2024-09-24 00:20:28,489 [ 200699 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy (runner:42, check_args_and_update_paths)
2024-09-24 00:20:28,489 [ 200699 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration (runner:90, check_args_and_update_paths)
2024-09-24 00:20:28,489 [ 200699 ] INFO : src dir is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy/src (runner:97, check_args_and_update_paths)
2024-09-24 00:20:28,489 [ 200699 ] INFO : base_configs_dir: /home/ubuntu/_work/_temp/test/git-repo-copy/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration  (runner:99, check_args_and_update_paths)
2024-09-24 00:20:28,489 [ 200699 ] INFO : Unknown image altinityinfra/dotnet-client (runner:357, <module>)
Running pytest container as: 'docker run   --rm --name clickhouse_integration_tests_g8i3jl --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos --volume=/run:/run/host:ro --mount type=bind,source=/home/ubuntu/_work/_temp/test/dockerd_volume_dir,target=/var/lib/docker -e DOCKER_HELPER_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_BASE_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_KERBERIZED_HADOOP_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_KERBEROS_KDC_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_JAVA_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_JS_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_PHP_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_NGINX_DAV_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553   -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_NEW_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_row_policy/test.py::test_some_users_without_policies test_row_policy/test.py::test_tags_with_db_and_table_names test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data test_row_policy/test.py::test_users_xml_is_readonly test_row_policy/test.py::test_with_prewhere test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage test_session_log/test.py::test_grpc_session test_session_log/test.py::test_mysql_session test_session_log/test.py::test_parallel_sessions test_session_log/test.py::test_postgres_session test_settings_constraints_distributed/test.py::test_insert_clamps_settings test_settings_constraints_distributed/test.py::test_select_clamps_settings test_storage_hudi/test.py::test_multiple_hudi_files test_storage_hudi/test.py::test_single_hudi_file test_storage_hudi/test.py::test_types test_storage_redis/test.py::test_create_table test_storage_redis/test.py::test_delete test_storage_redis/test.py::test_select_int test_storage_redis/test.py::test_simple_insert test_storage_redis/test.py::test_simple_select test_storage_redis/test.py::test_truncate test_storage_redis/test.py::test_update 'test_storage_s3_queue/test.py::test_delete_after_processing[ordered]' 'test_storage_s3_queue/test.py::test_delete_after_processing[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_file[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 'test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.part_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.query_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.text_log-0]' 'test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]' test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs 'test_throttling/test.py::test_backup_throttling[no_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]' test_throttling/test.py::test_backup_throttling_override 'test_throttling/test.py::test_read_throttling[no_local_throttling]' 'test_throttling/test.py::test_read_throttling[no_remote_throttling]' 'test_throttling/test.py::test_read_throttling[server_local_throttling]' 'test_throttling/test.py::test_read_throttling[server_remote_throttling]' 'test_throttling/test.py::test_read_throttling[user_local_throttling]' 'test_throttling/test.py::test_read_throttling[user_remote_throttling]' 'test_throttling/test.py::test_write_throttling[local_server_throttling]' 'test_throttling/test.py::test_write_throttling[local_user_throttling]' 'test_throttling/test.py::test_write_throttling[no_local_throttling]' 'test_throttling/test.py::test_write_throttling[no_remote_throttling]' 'test_throttling/test.py::test_write_throttling[server_remote_throttling]' 'test_throttling/test.py::test_write_throttling[user_remote_throttling]' test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 test_truncate_database/test_distributed.py::test_truncate_database_distributed test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_user_defined_object_persistence/test.py::test_persistence test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_zookeeper_config/test_secure.py::test_secure_connection  -vvv" altinityinfra/integration-tests-runner:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 '.
Start tests
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /ClickHouse/tests/integration
configfile: pytest.ini
plugins: timeout-2.2.0, repeat-0.9.3, order-1.0.1, xdist-3.5.0, random-0.2
timeout: 900.0s
timeout method: signal
timeout func_only: False
created: 10/10 workers
10 workers [95 items]

scheduling tests via LoadFileScheduling

test_throttling/test.py::test_backup_throttling[no_local_throttling] 
test_storage_s3_queue/test.py::test_delete_after_processing[ordered] 
test_settings_constraints_distributed/test.py::test_insert_clamps_settings 
test_storage_redis/test.py::test_create_table 
test_storage_hudi/test.py::test_multiple_hudi_files 
test_system_flush_logs/test.py::test_system_logs[system.metric_log-1] 
test_system_queries/test.py::test_DROP_DNS_CACHE 
test_session_log/test.py::test_grpc_session 
test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy 
test_row_policy/test.py::test_some_users_without_policies 
[gw3] [  1%] PASSED test_system_flush_logs/test.py::test_system_logs[system.metric_log-1] 
test_system_flush_logs/test.py::test_system_logs[system.part_log-1] 
[gw3] [  2%] PASSED test_system_flush_logs/test.py::test_system_logs[system.part_log-1] 
test_system_flush_logs/test.py::test_system_logs[system.query_log-1] 
[gw3] [  3%] PASSED test_system_flush_logs/test.py::test_system_logs[system.query_log-1] 
test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1] 
[gw6] [  4%] PASSED test_system_queries/test.py::test_DROP_DNS_CACHE 
test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS 
[gw3] [  5%] PASSED test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1] 
test_system_flush_logs/test.py::test_system_logs[system.text_log-0] 
[gw6] [  6%] PASSED test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS 
test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY 
[gw3] [  7%] PASSED test_system_flush_logs/test.py::test_system_logs[system.text_log-0] 
test_system_flush_logs/test.py::test_system_logs[system.trace_log-1] 
[gw3] [  8%] PASSED test_system_flush_logs/test.py::test_system_logs[system.trace_log-1] 
test_system_flush_logs/test.py::test_system_logs_non_empty_queue 
[gw3] [  9%] PASSED test_system_flush_logs/test.py::test_system_logs_non_empty_queue 
test_system_flush_logs/test.py::test_system_suspend 
[gw5] [ 10%] PASSED test_session_log/test.py::test_grpc_session 
test_session_log/test.py::test_mysql_session 
[gw0] [ 11%] PASSED test_throttling/test.py::test_backup_throttling[no_local_throttling] 
[gw6] [ 12%] PASSED test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY 
test_system_queries/test.py::test_system_flush_logs 
[gw5] [ 13%] PASSED test_session_log/test.py::test_mysql_session 
test_session_log/test.py::test_parallel_sessions 
[gw3] [ 14%] PASSED test_system_flush_logs/test.py::test_system_suspend 
test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage 
[gw5] [ 15%] PASSED test_session_log/test.py::test_parallel_sessions 
test_session_log/test.py::test_postgres_session 
[gw8] [ 16%] PASSED test_settings_constraints_distributed/test.py::test_insert_clamps_settings 
test_settings_constraints_distributed/test.py::test_select_clamps_settings 
test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] 
[gw0] [ 17%] PASSED test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] 
[gw5] [ 18%] PASSED test_session_log/test.py::test_postgres_session 
[gw6] [ 20%] PASSED test_system_queries/test.py::test_system_flush_logs 
[gw4] [ 21%] PASSED test_storage_redis/test.py::test_create_table 
test_storage_redis/test.py::test_delete 
test_version_update_after_mutation/test.py::test_mutate_and_upgrade 
test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 
[gw4] [ 22%] PASSED test_storage_redis/test.py::test_delete 
test_storage_redis/test.py::test_select_int 
test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] 
[gw0] [ 23%] PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] 
[gw4] [ 24%] PASSED test_storage_redis/test.py::test_select_int 
test_storage_redis/test.py::test_simple_insert 
[gw7] [ 25%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy 
test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list 
[gw4] [ 26%] PASSED test_storage_redis/test.py::test_simple_insert 
test_storage_redis/test.py::test_simple_select 
[gw4] [ 27%] PASSED test_storage_redis/test.py::test_simple_select 
test_storage_redis/test.py::test_truncate 
[gw7] [ 28%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list 
test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 
[gw4] [ 29%] PASSED test_storage_redis/test.py::test_truncate 
test_storage_redis/test.py::test_update 
[gw8] [ 30%] FAILED test_settings_constraints_distributed/test.py::test_select_clamps_settings 
[gw7] [ 31%] PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy 
[gw1] [ 32%] PASSED test_storage_s3_queue/test.py::test_delete_after_processing[ordered] 
test_storage_s3_queue/test.py::test_delete_after_processing[unordered] 
[gw4] [ 33%] PASSED test_storage_redis/test.py::test_update 
test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] 
[gw1] [ 34%] PASSED test_storage_s3_queue/test.py::test_delete_after_processing[unordered] 
test_storage_s3_queue/test.py::test_direct_select_file[ordered] 
[gw0] [ 35%] PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] 
test_transactions/test.py::test_rollback_unfinished_on_restart1 
[gw2] [ 36%] FAILED test_row_policy/test.py::test_some_users_without_policies 
[gw2] [ 36%] ERROR test_row_policy/test.py::test_some_users_without_policies 
test_row_policy/test.py::test_tags_with_db_and_table_names 
[gw2] [ 37%] FAILED test_row_policy/test.py::test_tags_with_db_and_table_names 
[gw2] [ 37%] ERROR test_row_policy/test.py::test_tags_with_db_and_table_names 
test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter 
[gw1] [ 38%] PASSED test_storage_s3_queue/test.py::test_direct_select_file[ordered] 
test_storage_s3_queue/test.py::test_direct_select_file[unordered] 
[gw2] [ 40%] FAILED test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter 
[gw2] [ 40%] ERROR test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter 
test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter 
[gw2] [ 41%] FAILED test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter 
[gw2] [ 41%] ERROR test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter 
test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data 
[gw2] [ 42%] FAILED test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data 
[gw2] [ 42%] ERROR test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data 
test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data 
[gw2] [ 43%] FAILED test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data 
[gw2] [ 43%] ERROR test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data 
test_row_policy/test.py::test_users_xml_is_readonly 
[gw3] [ 44%] SKIPPED test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage 
test_throttling/test.py::test_backup_throttling[server_local_throttling] 
[gw1] [ 45%] PASSED test_storage_s3_queue/test.py::test_direct_select_file[unordered] 
test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 
[gw1] [ 46%] PASSED test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 
test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 
[gw1] [ 47%] PASSED test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 
test_storage_s3_queue/test.py::test_failed_retry[ordered] 
[gw1] [ 48%] PASSED test_storage_s3_queue/test.py::test_failed_retry[ordered] 
test_storage_s3_queue/test.py::test_failed_retry[unordered] 
[gw1] [ 49%] PASSED test_storage_s3_queue/test.py::test_failed_retry[unordered] 
test_storage_s3_queue/test.py::test_max_set_age 
[gw0] [ 50%] PASSED test_throttling/test.py::test_backup_throttling[server_local_throttling] 
[gw2] [ 51%] FAILED test_row_policy/test.py::test_users_xml_is_readonly 
[gw2] [ 51%] ERROR test_row_policy/test.py::test_users_xml_is_readonly 
test_row_policy/test.py::test_with_prewhere 
[gw2] [ 52%] FAILED test_row_policy/test.py::test_with_prewhere 
[gw1] [ 53%] PASSED test_storage_s3_queue/test.py::test_max_set_age 
test_storage_s3_queue/test.py::test_max_set_size 
[gw1] [ 54%] PASSED test_storage_s3_queue/test.py::test_max_set_size 
test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 
test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] 
test_zookeeper_config/test_secure.py::test_secure_connection 
[gw1] [ 55%] PASSED test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 
test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered] 
[gw2] [ 55%] ERROR test_row_policy/test.py::test_with_prewhere 
[gw5] [ 56%] PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 
[gw8] [ 57%] PASSED test_transactions/test.py::test_rollback_unfinished_on_restart1 
test_transactions/test.py::test_rollback_unfinished_on_restart2 
test_storage_url_with_proxy/test.py::test_s3_with_proxy_list 
test_undrop_query/test.py::test_undrop_drop_and_undrop_loop 
[gw8] [ 58%] PASSED test_transactions/test.py::test_rollback_unfinished_on_restart2 
[gw0] [ 60%] PASSED test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] 
[gw3] [ 61%] PASSED test_storage_url_with_proxy/test.py::test_s3_with_proxy_list 
[gw1] [ 62%] PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered] 
test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 
test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] 
[gw6] [ 63%] PASSED test_version_update_after_mutation/test.py::test_mutate_and_upgrade 
test_version_update_after_mutation/test.py::test_upgrade_while_mutation 
[gw0] [ 64%] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] 
test_truncate_database/test_distributed.py::test_truncate_database_distributed 
test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] 
[gw6] [ 65%] PASSED test_version_update_after_mutation/test.py::test_upgrade_while_mutation 
[gw1] [ 66%] PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 
test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 
[gw0] [ 67%] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] 
test_user_defined_object_persistence/test.py::test_persistence 
[gw7] [ 68%] PASSED test_zookeeper_config/test_secure.py::test_secure_connection 
test_throttling/test.py::test_backup_throttling[user_local_throttling] 
[gw3] [ 69%] PASSED test_truncate_database/test_distributed.py::test_truncate_database_distributed 
[gw0] [ 70%] PASSED test_throttling/test.py::test_backup_throttling[user_local_throttling] 
[gw9] [ 71%] PASSED test_storage_hudi/test.py::test_multiple_hudi_files 
test_storage_hudi/test.py::test_single_hudi_file 
test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] 
[gw9] [ 72%] PASSED test_storage_hudi/test.py::test_single_hudi_file 
test_storage_hudi/test.py::test_types 
[gw6] [ 73%] FAILED test_user_defined_object_persistence/test.py::test_persistence 
[gw9] [ 74%] PASSED test_storage_hudi/test.py::test_types 
[gw1] [ 75%] PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 
test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 
[gw0] [ 76%] PASSED test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] 
test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] 
[gw1] [ 77%] PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 
test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 
[gw0] [ 78%] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] 
test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] 
[gw0] [ 80%] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] 
[gw5] [ 81%] PASSED test_undrop_query/test.py::test_undrop_drop_and_undrop_loop 
test_throttling/test.py::test_backup_throttling_override 
[gw1] [ 82%] PASSED test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 
test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 
[gw1] [ 83%] PASSED test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 
test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 
test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access 
[gw1] [ 84%] PASSED test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 
test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 
[gw1] [ 85%] PASSED test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 
[gw5] [ 86%] PASSED test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access 
[gw0] [ 87%] PASSED test_throttling/test.py::test_backup_throttling_override 
test_throttling/test.py::test_read_throttling[no_local_throttling] 
[gw0] [ 88%] PASSED test_throttling/test.py::test_read_throttling[no_local_throttling] 
test_throttling/test.py::test_read_throttling[no_remote_throttling] 
[gw0] [ 89%] PASSED test_throttling/test.py::test_read_throttling[no_remote_throttling] 
test_throttling/test.py::test_read_throttling[server_local_throttling] 
[gw0] [ 90%] PASSED test_throttling/test.py::test_read_throttling[server_local_throttling] 
test_throttling/test.py::test_read_throttling[server_remote_throttling] 
[gw0] [ 91%] PASSED test_throttling/test.py::test_read_throttling[server_remote_throttling] 
test_throttling/test.py::test_read_throttling[user_local_throttling] 
[gw0] [ 92%] PASSED test_throttling/test.py::test_read_throttling[user_local_throttling] 
test_throttling/test.py::test_read_throttling[user_remote_throttling] 
[gw0] [ 93%] PASSED test_throttling/test.py::test_read_throttling[user_remote_throttling] 
test_throttling/test.py::test_write_throttling[local_server_throttling] 
[gw0] [ 94%] PASSED test_throttling/test.py::test_write_throttling[local_server_throttling] 
test_throttling/test.py::test_write_throttling[local_user_throttling] 
[gw0] [ 95%] PASSED test_throttling/test.py::test_write_throttling[local_user_throttling] 
test_throttling/test.py::test_write_throttling[no_local_throttling] 
[gw0] [ 96%] PASSED test_throttling/test.py::test_write_throttling[no_local_throttling] 
test_throttling/test.py::test_write_throttling[no_remote_throttling] 
[gw0] [ 97%] PASSED test_throttling/test.py::test_write_throttling[no_remote_throttling] 
test_throttling/test.py::test_write_throttling[server_remote_throttling] 
[gw0] [ 98%] PASSED test_throttling/test.py::test_write_throttling[server_remote_throttling] 
test_throttling/test.py::test_write_throttling[user_remote_throttling] 
[gw0] [100%] PASSED test_throttling/test.py::test_write_throttling[user_remote_throttling] 

==================================== ERRORS ====================================
____________ ERROR at teardown of test_some_users_without_policies _____________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
---------------------------- Captured stdout setup -----------------------------
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
---------------------------- Captured stderr setup -----------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
------------------------------ Captured log setup ------------------------------
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Stdout:1 (cluster.py:113, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : No running containers (conftest.py:60, cleanup_environment)
2024-09-24 00:20:32 [ 367 ] DEBUG : Pruning Docker networks (conftest.py:62, cleanup_environment)
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:['docker network prune --force'] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:113, run_and_check)
2024-09-24 00:20:32 [ 367 ] INFO : Running tests in /ClickHouse/tests/integration/test_row_policy/test.py (cluster.py:2637, start)
2024-09-24 00:20:32 [ 367 ] DEBUG : Cluster start called. is_up=False (cluster.py:2644, start)
2024-09-24 00:20:32 [ 367 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 42
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker networks for project roottestrowpolicy are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker containers for project roottestrowpolicy are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker volumes for project roottestrowpolicy are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:20:32 [ 367 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 42
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker networks for project roottestrowpolicy are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker containers for project roottestrowpolicy are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:20:33 [ 367 ] DEBUG : Docker volumes for project roottestrowpolicy are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:docker container list --all --filter name='^/roottestrowpolicy_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : No running containers for project: roottestrowpolicy (cluster.py:782, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Stderr:Error response from daemon: a prune operation is already running (cluster.py:115, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Stdout:1 (cluster.py:113, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup directory for instance: node (cluster.py:2657, start)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4309, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for common tests configuration (cluster.py:4314, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy common configuration from helpers (cluster.py:4334, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Generate and write macros file (cluster.py:4353, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_row_policy/configs/config.d/remote_servers.xml'] to /ClickHouse/tests/integration/test_row_policy/_instances_0/node/configs/config.d (cluster.py:4383, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node/database (cluster.py:4400, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node/logs (cluster.py:4411, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4487, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup directory for instance: node2 (cluster.py:2657, start)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4309, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for common tests configuration (cluster.py:4314, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy common configuration from helpers (cluster.py:4334, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Generate and write macros file (cluster.py:4353, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_row_policy/configs/config.d/remote_servers.xml'] to /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/configs/config.d (cluster.py:4383, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/database (cluster.py:4400, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/logs (cluster.py:4411, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4487, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_row_policy/_instances_0/.env (cluster.py:78, _create_env_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:546, _make_request)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'pull'] (cluster.py:105, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Setup ZooKeeper (cluster.py:2699, start)
2024-09-24 00:20:44 [ 367 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination'] (cluster.py:2700, start)
2024-09-24 00:20:44 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:105, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /compose/docker_compose_keeper.yml (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:docker-py version: <module 'docker.version' from '/usr/local/lib/python3.10/dist-packages/docker/version.py'> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-119-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': '1.7.22', 'Details': {'GitCommit': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c'}}, {'Name': 'runc', 'Version': '1.1.14', 'Details': {'GitCommit': 'v1.1.14-0-g2c9f560'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-119-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:                      'ID': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Containers': 2, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.network.ensure: Creating network "roottestrowpolicy_default" with the default driver (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='roottestrowpolicy_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': 'da700a446e5d138cd7a1476f2cce92496fdc1a4343f493d9f229838c2717d45e', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {<Service: zoo3>, <Service: zoo2>, <Service: zoo1>} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo3> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo2> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo3_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo3', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo3', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo1> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo2', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo2', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 01c5d05649c34834ec7a89763669f921de701a6ee114c6a489f91553f7c07658 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/lib/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '01c5d05649c34834ec7a89763669f921de701a6ee114c6a489f91553f7c07658'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 3f50bb25fb81f816e4dce31f9d66f0e9cc9c3efa697fed552a15d3029592ca9b (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/lib/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '3f50bb25fb81f816e4dce31f9d66f0e9cc9c3efa697fed552a15d3029592ca9b'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo1_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo1', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo1', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 8aa383da29ffec3344f661d9e6b5b6fb9f0a0d021963da648b0982ab8b451851 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/log/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '8aa383da29ffec3344f661d9e6b5b6fb9f0a0d021963da648b0982ab8b451851'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', 'roottestrowpolicy_default', aliases=['zoo3', '43c2a2b5df56'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', 'roottestrowpolicy_default', aliases=['zoo2', '620722d4c0be'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', 'roottestrowpolicy_default', aliases=['49e31cff24df', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo3', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo3_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo3> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo2', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo2> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo1', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo1_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo1> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Wait ZooKeeper to start (cluster.py:2337, wait_zookeeper_to_start)
2024-09-24 00:20:46 [ 367 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:46 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:46 [ 367 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.8.4, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:46 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:46 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:46 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:46 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:47 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:47 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:47 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:47 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:48 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:48 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:48 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:48 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:51 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.8.2, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:51 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.8.3, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.3(172.16.8.3):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : All instances of ZooKeeper started (cluster.py:2345, wait_zookeeper_to_start)
2024-09-24 00:20:51 [ 367 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_row_policy/_instances_0/.env --project-name roottestrowpolicy --file /ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml --file /compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml up -d --no-recreate') (cluster.py:2980, start)
2024-09-24 00:20:51 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:105, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : ClickHouse instance created (cluster.py:2988, start)
2024-09-24 00:20:52 [ 367 ] DEBUG : get_instance_ip instance_name=node (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : Waiting for ClickHouse start in node, ip: 172.16.8.6... (cluster.py:2995, start)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : ClickHouse node started (cluster.py:2999, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.8.5... (cluster.py:2995, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/2a77f6b1142771672473e46b34a80545b31bd944363e4f8292aea651670f699a/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : ClickHouse node2 started (cluster.py:2999, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : Executing query 
                CREATE DATABASE mydb;

                CREATE TABLE mydb.filtered_table1 (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table1 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.table (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.table values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.filtered_table2 (a UInt8, b UInt8, c UInt8, d UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table2 values (0, 0, 0, 0), (1, 2, 3, 4), (4, 3, 2, 1), (0, 0, 6, 0);

                CREATE TABLE mydb.filtered_table3 (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table3 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.`.filtered_table4` (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.`.filtered_t on node (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query 
                CREATE DATABASE mydb;

                CREATE TABLE mydb.filtered_table1 (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table1 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.table (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.table values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.filtered_table2 (a UInt8, b UInt8, c UInt8, d UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table2 values (0, 0, 0, 0), (1, 2, 3, 4), (4, 3, 2, 1), (0, 0, 6, 0);

                CREATE TABLE mydb.filtered_table3 (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table3 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.`.filtered_table4` (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.`.filtered_t on node2 (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query INSERT INTO mydb.local values (2, 0), (2, 1), (1, 0), (1, 1) on node (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query INSERT INTO mydb.local values (3, 0), (3, 1), (1, 0), (1, 1) on node2 (cluster.py:3423, query)
------------------------------ Captured log call -------------------------------
2024-09-24 00:20:55 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:55 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SYSTEM RELOAD CONFIG on node (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node2_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:56 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node2_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SYSTEM RELOAD CONFIG on node2 (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SHOW POLICIES on node (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query CREATE USER X, Y on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query GRANT SELECT ON mydb.filtered_table1 TO X, Y on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query CREATE POLICY pA ON mydb.filtered_table1 FOR SELECT USING a >= b AS permissive TO X on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query SELECT * FROM mydb.filtered_table1 on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query SELECT * FROM mydb.filtered_table1 on node (cluster.py:3423, query)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:12 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:12 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:12 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
____________ ERROR at teardown of test_tags_with_db_and_table_names ____________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8dGFibGUgbmFtZT0nZmlsdGVyZWRfdGFibGUyJz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jICZndDsgZCArIDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L3RhYmxlPgogICAgICAgICAgICAgICAgPC9teWRiPgoKICAgICAgICAgICAgICAgIDxteWRiPgogICAgICAgICAgICAgICAgICAgIDx0YWJsZT4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5hID0gMDwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CgogICAgICAgICAgICAgICAgICAgIDx0YWJsZSBuYW1lPScuZmlsdGVyZWRfdGFibGU0Jz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMjwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CiAgICAgICAgICAgICAgICA8L215ZGI+CgogICAgICAgICAgICAgICAgPGRhdGFiYXNlIG5hbWU9J215ZGInPgogICAgICAgICAgICAgICAgICAgIDxmaWx0ZXJlZF90YWJsZTM+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YyA9IDA8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgIDwvZGF0YWJhc2U+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8dGFibGUgbmFtZT0nZmlsdGVyZWRfdGFibGUyJz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jICZndDsgZCArIDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L3RhYmxlPgogICAgICAgICAgICAgICAgPC9teWRiPgoKICAgICAgICAgICAgICAgIDxteWRiPgogICAgICAgICAgICAgICAgICAgIDx0YWJsZT4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5hID0gMDwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CgogICAgICAgICAgICAgICAgICAgIDx0YWJsZSBuYW1lPScuZmlsdGVyZWRfdGFibGU0Jz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMjwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CiAgICAgICAgICAgICAgICA8L215ZGI+CgogICAgICAgICAgICAgICAgPGRhdGFiYXNlIG5hbWU9J215ZGInPgogICAgICAgICAgICAgICAgICAgIDxmaWx0ZXJlZF90YWJsZTM+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YyA9IDA8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgIDwvZGF0YWJhc2U+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_ ERROR at teardown of test_throwif_error_in_prewhere_with_same_condition_as_filter _
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_ ERROR at teardown of test_throwif_error_in_where_with_same_condition_as_filter _
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_ ERROR at teardown of test_throwif_in_prewhere_doesnt_expose_restricted_data __
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
___ ERROR at teardown of test_throwif_in_where_doesnt_expose_restricted_data ___
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:15 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:15 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:15 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:15 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_______________ ERROR at teardown of test_users_xml_is_readonly ________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:15 [ 367 ] DEBUG : Executing query DROP POLICY default ON mydb.filtered_table1 on node (cluster.py:3495, query_and_get_error)
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:25 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:25 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:25 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:25 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
___________________ ERROR at teardown of test_with_prewhere ____________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    @pytest.fixture(autouse=True)
    def reset_policies():
        try:
            yield
        finally:
>           copy_policy_xml("normal_filters.xml")

test_row_policy/test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:26 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:26 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
--------------------------- Captured stderr teardown ---------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:26 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:26 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUxPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgPSAxPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTE+CgogICAgICAgICAgICAgICAgICAgIDwhLS0gQ29tcGxleCBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgKyBiICZsdDsgMSBvciBjIC0gZCAmZ3Q7IDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMj4KCiAgICAgICAgICAgICAgICAgICAgPCEtLSBGaWx0ZXIgd2l0aCBBTElBUyBjb2x1bW4gLS0+CiAgICAgICAgICAgICAgICAgICAgPGZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMTwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvZmlsdGVyZWRfdGFibGUzPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2RlZmF1bHQ+CiAgICAgICAgPGFub3RoZXI+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8bG9jYWw+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YSA9IDE8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2xvY2FsPgogICAgICAgICAgICAgICAgPC9teWRiPgogICAgICAgICAgICA8L2RhdGFiYXNlcz4KICAgICAgICA8L2Fub3RoZXI+CiAgICA8L3VzZXJzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:105, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_node2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo1_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo2_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo3_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_node2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo1_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo2_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Stderr:Stopping roottestrowpolicy_zoo3_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_row_policy/_instances_0/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_row_policy/_instances_0/node/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:21:31 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'down', '--volumes'] (cluster.py:105, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_node_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_node2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo1_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo2_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo3_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo3_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_node2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo1_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_zoo2_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing roottestrowpolicy_node_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stderr:Removing network roottestrowpolicy_default (cluster.py:115, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 41
  Running: 39
  Paused: 0
  Stopped: 2
 Images: 43
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:21:32 [ 367 ] DEBUG : Docker networks for project roottestrowpolicy are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:21:32 [ 367 ] DEBUG : Docker containers for project roottestrowpolicy are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:21:32 [ 367 ] DEBUG : Docker volumes for project roottestrowpolicy are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:21:32 [ 367 ] DEBUG : Command:docker container list --all --filter name='^/roottestrowpolicy_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : No running containers for project: roottestrowpolicy (cluster.py:782, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2024-09-24 00:21:32 [ 367 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:21:32 [ 367 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:21:33 [ 367 ] DEBUG : Stdout:9 (cluster.py:113, run_and_check)
=================================== FAILURES ===================================
_________________________ test_select_clamps_settings __________________________
[gw8] linux -- Python 3.10.12 /usr/bin/python3

    def test_select_clamps_settings():
        for node in [node1, node2]:
            node.query(
                "CREATE TABLE sometable_select (date Date, id UInt32, value Int32) ENGINE = MergeTree() ORDER BY id;"
            )
            node.query("INSERT INTO sometable_select VALUES (toDate('2010-01-10'), 1, 1)")
    
        distributed.query(
            "CREATE TABLE proxy_select (date Date, id UInt32, value Int32) ENGINE = Distributed(test_cluster, default, sometable_select, toUInt64(date));"
        )
    
        distributed.query(
            "CREATE USER normal DEFAULT ROLE admin SETTINGS max_memory_usage = 80000000"
        )
        distributed.query(
            "CREATE USER wasteful DEFAULT ROLE admin SETTINGS max_memory_usage = 2000000000"
        )
        distributed.query("CREATE USER readonly DEFAULT ROLE admin SETTINGS readonly = 1")
        node1.query(
            "ALTER USER shard SETTINGS max_memory_usage = 50000000 MIN 11111111 MAX 99999999"
        )
        node2.query("ALTER USER shard SETTINGS readonly = 1")
    
        # Check that shards doesn't throw exceptions on constraints violation
        query = "SELECT COUNT() FROM proxy_select"
        assert distributed.query(query) == "2\n"
        assert distributed.query(query, user="normal") == "2\n"
        assert distributed.query(query, user="wasteful") == "2\n"
        assert distributed.query(query, user="readonly") == "2\n"
    
        assert (
            distributed.query(query, settings={"max_memory_usage": 40000000, "readonly": 2})
            == "2\n"
        )
        assert (
            distributed.query(
                query, settings={"max_memory_usage": 3000000000, "readonly": 2}
            )
            == "2\n"
        )
    
        query = "SELECT COUNT() FROM remote('node{1,2}', 'default', 'sometable_select')"
        assert distributed.query(query) == "2\n"
        assert distributed.query(query, user="normal") == "2\n"
        assert distributed.query(query, user="wasteful") == "2\n"
    
        # Check that shards clamp passed settings.
        query = "SELECT hostName() as host, name, value FROM shard_settings WHERE name = 'max_memory_usage' OR name = 'readonly' ORDER BY host, name, value"
>       assert (
            distributed.query(query) == "node1\tmax_memory_usage\t50000000\n"
            "node1\treadonly\t0\n"
            "node2\tmax_memory_usage\t0\n"
            "node2\treadonly\t1\n"
        )
E       AssertionError: assert 'distributed\tmax_memory_usage\t50000000\ndistributed\treadonly\t0\ndistributed\tmax_memory_usage\t0\ndistributed\treadonly\t1\n' == 'node1\tmax_memory_usage\t50000000\nnode1\treadonly\t0\nnode2\tmax_memory_usage\t0\nnode2\treadonly\t1\n'
E         - node1	max_memory_usage	50000000
E         ? --  ^
E         + distributed	max_memory_usage	50000000
E         ?  ++++++++ ^
E         - node1	readonly	0
E         + distributed	readonly	0
E         - node2	max_memory_usage	0
E         ? --  ^
E         + distributed	max_memory_usage	0
E         ?  ++++++++ ^
E         - node2	readonly	1
E         + distributed	readonly	1

test_settings_constraints_distributed/test.py:111: AssertionError
------------------------------ Captured log setup ------------------------------
2024-09-24 00:20:57 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:57 [ 385 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:20:57 [ 385 ] DEBUG : Stdout:    PID TTY          TIME CMD (cluster.py:113, run_and_check)
2024-09-24 00:20:57 [ 385 ] DEBUG : Stdout:    343 ?        00:00:02 clickhouse (cluster.py:113, run_and_check)
2024-09-24 00:20:57 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill  clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:57 [ 385 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', 'pkill  clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:20:57 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:57 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:20:57 [ 385 ] DEBUG : Stdout:343 (cluster.py:113, run_and_check)
2024-09-24 00:20:58 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:58 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:20:58 [ 385 ] DEBUG : Stdout:343 (cluster.py:113, run_and_check)
2024-09-24 00:20:59 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:59 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:20:59 [ 385 ] DEBUG : Stdout:343 (cluster.py:113, run_and_check)
2024-09-24 00:21:00 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:00 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:21:00 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:00 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:21:00 [ 385 ] DEBUG : No clickhouse process running. Start new one. (cluster.py:3775, start_clickhouse)
2024-09-24 00:21:00 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:00 [ 385 ] DEBUG : Command:['docker', 'exec', '-u', '0', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:105, run_and_check)
2024-09-24 00:21:02 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:02 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:21:02 [ 385 ] DEBUG : Stdout:729 (cluster.py:113, run_and_check)
2024-09-24 00:21:02 [ 385 ] DEBUG : Clickhouse process running. (cluster.py:3783, start_clickhouse)
2024-09-24 00:21:02 [ 385 ] DEBUG : run container_id:roottestsettingsconstraintsdistributed_distributed_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:02 [ 385 ] DEBUG : Command:['docker', 'exec', 'roottestsettingsconstraintsdistributed_distributed_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:21:02 [ 385 ] DEBUG : Stdout:729 (cluster.py:113, run_and_check)
2024-09-24 00:21:02 [ 385 ] DEBUG : Executing query select 20 on distributed (cluster.py:3423, query)
2024-09-24 00:21:03 [ 385 ] DEBUG : Executing query select 20 on distributed (cluster.py:3423, query)
2024-09-24 00:21:03 [ 385 ] DEBUG : Executing query select 20 on distributed (cluster.py:3423, query)
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:04 [ 385 ] DEBUG : Executing query CREATE TABLE sometable_select (date Date, id UInt32, value Int32) ENGINE = MergeTree() ORDER BY id; on node1 (cluster.py:3423, query)
2024-09-24 00:21:04 [ 385 ] DEBUG : Executing query INSERT INTO sometable_select VALUES (toDate('2010-01-10'), 1, 1) on node1 (cluster.py:3423, query)
2024-09-24 00:21:04 [ 385 ] DEBUG : Executing query CREATE TABLE sometable_select (date Date, id UInt32, value Int32) ENGINE = MergeTree() ORDER BY id; on node2 (cluster.py:3423, query)
2024-09-24 00:21:04 [ 385 ] DEBUG : Executing query INSERT INTO sometable_select VALUES (toDate('2010-01-10'), 1, 1) on node2 (cluster.py:3423, query)
2024-09-24 00:21:04 [ 385 ] DEBUG : Executing query CREATE TABLE proxy_select (date Date, id UInt32, value Int32) ENGINE = Distributed(test_cluster, default, sometable_select, toUInt64(date)); on distributed (cluster.py:3423, query)
2024-09-24 00:21:05 [ 385 ] DEBUG : Executing query CREATE USER normal DEFAULT ROLE admin SETTINGS max_memory_usage = 80000000 on distributed (cluster.py:3423, query)
2024-09-24 00:21:05 [ 385 ] DEBUG : Executing query CREATE USER wasteful DEFAULT ROLE admin SETTINGS max_memory_usage = 2000000000 on distributed (cluster.py:3423, query)
2024-09-24 00:21:05 [ 385 ] DEBUG : Executing query CREATE USER readonly DEFAULT ROLE admin SETTINGS readonly = 1 on distributed (cluster.py:3423, query)
2024-09-24 00:21:05 [ 385 ] DEBUG : Executing query ALTER USER shard SETTINGS max_memory_usage = 50000000 MIN 11111111 MAX 99999999 on node1 (cluster.py:3423, query)
2024-09-24 00:21:05 [ 385 ] DEBUG : Executing query ALTER USER shard SETTINGS readonly = 1 on node2 (cluster.py:3423, query)
2024-09-24 00:21:06 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:06 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:06 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:06 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:07 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:07 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM proxy_select on distributed (cluster.py:3423, query)
2024-09-24 00:21:07 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM remote('node{1,2}', 'default', 'sometable_select') on distributed (cluster.py:3423, query)
2024-09-24 00:21:07 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM remote('node{1,2}', 'default', 'sometable_select') on distributed (cluster.py:3423, query)
2024-09-24 00:21:07 [ 385 ] DEBUG : Executing query SELECT COUNT() FROM remote('node{1,2}', 'default', 'sometable_select') on distributed (cluster.py:3423, query)
2024-09-24 00:21:08 [ 385 ] DEBUG : Executing query SELECT hostName() as host, name, value FROM shard_settings WHERE name = 'max_memory_usage' OR name = 'readonly' ORDER BY host, name, value on distributed (cluster.py:3423, query)
--------------------------- Captured stderr teardown ---------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:21:09 [ 385 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/.env', '--project-name', 'roottestsettingsconstraintsdistributed', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/distributed/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:105, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_node2_1       ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_node1_1       ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_distributed_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_distributed_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_node1_1       ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Stderr:Stopping roottestsettingsconstraintsdistributed_node2_1       ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node1/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node2/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/distributed/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/distributed/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:21:11 [ 385 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/.env', '--project-name', 'roottestsettingsconstraintsdistributed', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_settings_constraints_distributed/_instances_0/distributed/docker-compose.yml', 'down', '--volumes'] (cluster.py:105, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_node2_1       ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_node1_1       ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_distributed_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_node2_1       ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_node1_1       ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing roottestsettingsconstraintsdistributed_distributed_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stderr:Removing network roottestsettingsconstraintsdistributed_default (cluster.py:115, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 37
  Running: 34
  Paused: 0
  Stopped: 3
 Images: 42
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:21:12 [ 385 ] DEBUG : Docker networks for project roottestsettingsconstraintsdistributed are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:21:12 [ 385 ] DEBUG : Docker containers for project roottestsettingsconstraintsdistributed are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:21:12 [ 385 ] DEBUG : Docker volumes for project roottestsettingsconstraintsdistributed are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:21:12 [ 385 ] DEBUG : Command:docker container list --all --filter name='^/roottestsettingsconstraintsdistributed_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : No running containers for project: roottestsettingsconstraintsdistributed (cluster.py:782, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:21:12 [ 385 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:21:12 [ 385 ] DEBUG : Stdout:12 (cluster.py:113, run_and_check)
_______________________ test_some_users_without_policies _______________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_some_users_without_policies():
        copy_policy_xml("no_filters.xml")
        assert node.query("SHOW POLICIES") == ""
        node.query("CREATE USER X, Y")
        node.query("GRANT SELECT ON mydb.filtered_table1 TO X, Y")
    
        # permissive a >= b for X, none for Y
        node.query(
            "CREATE POLICY pA ON mydb.filtered_table1 FOR SELECT USING a >= b AS permissive TO X"
        )
        assert node.query("SELECT * FROM mydb.filtered_table1", user="X") == TSV(
            [[0, 0], [1, 0], [1, 1]]
        )
>       assert node.query("SELECT * FROM mydb.filtered_table1", user="Y") == TSV(
            [[0, 0], [0, 1], [1, 0], [1, 1]]
        )

test_row_policy/test.py:757: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
helpers/cluster.py:3424: in query
    return self.client.query(
helpers/client.py:36: in wrap
    return func(self, *args, **kwargs)
helpers/client.py:74: in query
    ).get_answer()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <helpers.client.CommandRequest object at 0x7f2638334e50>

    def get_answer(self):
        self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT)
        self.stdout_file.seek(0)
        self.stderr_file.seek(0)
    
        stdout = self.stdout_file.read().decode("utf-8", errors="replace")
        stderr = self.stderr_file.read().decode("utf-8", errors="replace")
    
        if (
            self.timer is not None
            and not self.process_finished_before_timeout
            and not self.ignore_error
        ):
            logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}")
            raise QueryTimeoutExceedException("Client timed out!")
    
        if (
            self.process.returncode != 0 or self.remove_trash_from_stderr(stderr)
        ) and not self.ignore_error:
>           raise QueryRuntimeException(
                "Client failed! Return code: {}, stderr: {}".format(
                    self.process.returncode, stderr
                ),
                self.process.returncode,
                stderr,
            )
E           helpers.client.QueryRuntimeException: Client failed! Return code: 32, stderr: [node] 2024.09.24 00:20:58.143792 [ 284 ] <Fatal> BaseDaemon: ########################################
E           [node] 2024.09.24 00:20:58.143989 [ 284 ] <Fatal> BaseDaemon: (version 23.8.16.41.altinitystable (altinity build), build id: E44628158B46C503106597583147763472F4E32F, git hash: 9992f5eba3591142ff413d31cb83a7bf4f0e2553) (from thread 8) (query_id: e65bb4f4-5f59-4952-85be-b86060756015) (query: SELECT * FROM mydb.filtered_table1) Received signal Segmentation fault (11)
E           [node] 2024.09.24 00:20:58.144068 [ 284 ] <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
E           [node] 2024.09.24 00:20:58.144138 [ 284 ] <Fatal> BaseDaemon: Stack trace: 0x0000559fa4d6b9cd 0x0000559fa5544615 0x00007f17f6f4c520 0x0000559fb88b297c 0x0000559fb8d42f27 0x0000559fb8d66bee 0x0000559fb8d5a9aa 0x0000559fb8d0baf6 0x0000559fb8d05e67 0x0000559fb8d03ab2 0x0000559fb97f167d 0x0000559fb97e9301 0x0000559fbc5da2fd 0x0000559fbc6170d0 0x0000559fc53b908f 0x0000559fc53b9e57 0x0000559fc589967c 0x0000559fc5892728 0x00007f17f6f9eac3 0x00007f17f7030850
E           [node] 2024.09.24 00:20:58.217461 [ 284 ] <Fatal> BaseDaemon: 0.1. inlined from ./build_docker/./src/Common/StackTrace.cpp:286: StackTrace::tryCapture()
E           [node] 2024.09.24 00:20:58.217615 [ 284 ] <Fatal> BaseDaemon: 0. ./build_docker/./src/Common/StackTrace.cpp:258: StackTrace::StackTrace(ucontext_t const&) @ 0x000000001d65e9cd in /usr/bin/clickhouse
E           [node] 2024.09.24 00:20:58.458382 [ 284 ] <Fatal> BaseDaemon: 1.1. inlined from ./build_docker/./src/IO/WriteHelpers.h:88: void DB::writePODBinary<int>(int const&, DB::WriteBuffer&)
E           [node] 2024.09.24 00:20:58.458532 [ 284 ] <Fatal> BaseDaemon: 1.2. inlined from ./build_docker/./src/IO/WriteHelpers.h:881: void DB::writeBinary<int>(int const&, DB::WriteBuffer&)
E           [node] 2024.09.24 00:20:58.458628 [ 284 ] <Fatal> BaseDaemon: 1. ./build_docker/./src/Daemon/BaseDaemon.cpp:152: signalHandler(int, siginfo_t*, void*) @ 0x000000001de37615 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:20:58.458725 [ 284 ] <Fatal> BaseDaemon: 2. ? @ 0x00007f17f6f4c520 in ?
E           [node] 2024.09.24 00:20:58.746813 [ 284 ] <Fatal> BaseDaemon: 3.1. inlined from ./build_docker/./src/Analyzer/QueryTreeBuilder.cpp:123: QueryTreeBuilder
E           [node] 2024.09.24 00:20:58.746949 [ 284 ] <Fatal> BaseDaemon: 3. ./build_docker/./src/Analyzer/QueryTreeBuilder.cpp:1029: DB::buildQueryTree(std::shared_ptr<DB::IAST>, std::shared_ptr<DB::Context const>) @ 0x00000000311a597c in /usr/bin/clickhouse
E           [node] 2024.09.24 00:20:59.039431 [ 284 ] <Fatal> BaseDaemon: 4. ./build_docker/./src/Planner/Utils.cpp:0: DB::buildFilterInfo(std::shared_ptr<DB::IAST>, std::shared_ptr<DB::IQueryTreeNode> const&, std::shared_ptr<DB::PlannerContext>&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>>) @ 0x0000000031635f27 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:20:59.530300 [ 284 ] <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__hash_table:1473: ~__hash_table
E           [node] 2024.09.24 00:20:59.530457 [ 284 ] <Fatal> BaseDaemon: 5.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/unordered_set:615: ~unordered_set
E           [node] 2024.09.24 00:20:59.530557 [ 284 ] <Fatal> BaseDaemon: 5.3. inlined from ./build_docker/./src/Planner/PlannerJoinTree.cpp:447: DB::(anonymous namespace)::buildRowPolicyFilterIfNeeded(std::shared_ptr<DB::IStorage> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::PlannerContext>&)
E           [node] 2024.09.24 00:20:59.530658 [ 284 ] <Fatal> BaseDaemon: 5. ./build_docker/./src/Planner/PlannerJoinTree.cpp:730: DB::(anonymous namespace)::buildQueryPlanForTableExpression(std::shared_ptr<DB::IQueryTreeNode>, DB::SelectQueryInfo const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::PlannerContext>&, bool, bool) @ 0x0000000031659bee in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:00.076833 [ 284 ] <Fatal> BaseDaemon: 6. ./build_docker/./src/Planner/PlannerJoinTree.cpp:1450: DB::buildJoinTreeQueryPlan(std::shared_ptr<DB::IQueryTreeNode> const&, DB::SelectQueryInfo const&, DB::SelectQueryOptions&, std::unordered_set<String, std::hash<String>, std::equal_to<String>, std::allocator<String>> const&, std::shared_ptr<DB::PlannerContext>&) @ 0x000000003164d9aa in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:00.371678 [ 284 ] <Fatal> BaseDaemon: 7. ./build_docker/./src/Planner/Planner.cpp:1258: DB::Planner::buildPlanForQueryNode() @ 0x00000000315feaf6 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:00.652122 [ 284 ] <Fatal> BaseDaemon: 8. ./build_docker/./src/Planner/Planner.cpp:1090: DB::Planner::buildQueryPlanIfNeeded() @ 0x00000000315f8e67 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:00.768505 [ 284 ] <Fatal> BaseDaemon: 9.1. inlined from ./build_docker/./src/Planner/Planner.h:44: DB::Planner::getQueryPlan()
E           [node] 2024.09.24 00:21:00.768640 [ 284 ] <Fatal> BaseDaemon: 9. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:235: DB::InterpreterSelectQueryAnalyzer::getQueryPlan() @ 0x00000000315f6ab2 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:01.162023 [ 284 ] <Fatal> BaseDaemon: 10. ./build_docker/./src/Interpreters/executeQuery.cpp:1041: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320e467d in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:01.602422 [ 284 ] <Fatal> BaseDaemon: 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320dc301 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:01.870455 [ 284 ] <Fatal> BaseDaemon: 12. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ecd2fd in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.492947 [ 284 ] <Fatal> BaseDaemon: 13. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f0a0d0 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.506808 [ 284 ] <Fatal> BaseDaemon: 14. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcac08f in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.529818 [ 284 ] <Fatal> BaseDaemon: 15.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: std::default_delete<Poco::Net::TCPServerConnection>::operator()[abi:v15000](Poco::Net::TCPServerConnection*) const
E           [node] 2024.09.24 00:21:02.530944 [ 284 ] <Fatal> BaseDaemon: 15.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:305: std::unique_ptr<Poco::Net::TCPServerConnection, std::default_delete<Poco::Net::TCPServerConnection>>::reset[abi:v15000](Poco::Net::TCPServerConnection*)
E           [node] 2024.09.24 00:21:02.531069 [ 284 ] <Fatal> BaseDaemon: 15.3. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:259: ~unique_ptr
E           [node] 2024.09.24 00:21:02.531233 [ 284 ] <Fatal> BaseDaemon: 15. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:116: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcace57 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.562878 [ 284 ] <Fatal> BaseDaemon: 16. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e18c67c in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.588449 [ 284 ] <Fatal> BaseDaemon: 17. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e185728 in /usr/bin/clickhouse
E           [node] 2024.09.24 00:21:02.588639 [ 284 ] <Fatal> BaseDaemon: 18. ? @ 0x00007f17f6f9eac3 in ?
E           [node] 2024.09.24 00:21:02.588732 [ 284 ] <Fatal> BaseDaemon: 19. ? @ 0x00007f17f7030850 in ?
E           [node] 2024.09.24 00:21:04.003113 [ 284 ] <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 1E52029798AD9E6464A7305FA1863306)
E           [node] 2024.09.24 00:21:08.469268 [ 284 ] <Fatal> BaseDaemon: You are using an Altinity Stable Build. Please log issues at https://github.com/Altinity/ClickHouse/issues. Thank you!
E           [node] 2024.09.24 00:21:08.469495 [ 284 ] <Fatal> BaseDaemon: Changed settings: any_join_distinct_right_table_keys = true, allow_experimental_analyzer = true, optimize_on_insert = false
E           Error on processing query: Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from 172.16.8.6:9000. (ATTEMPT_TO_READ_AFTER_EOF), Stack trace (when copying this message, always include the lines below):
E           
E           0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e077d33 in /clickhouse
E           1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5c5dd0 in /clickhouse
E           2. DB::Exception::Exception<char const (&) [26]>(int, char const (&) [26]) @ 0x000000000c5e7649 in /clickhouse
E           3. ./build_docker/./src/IO/VarInt.cpp:13: DB::throwReadAfterEOF() @ 0x000000001d755ffe in /clickhouse
E           4. ./build_docker/./src/QueryPipeline/ProfileInfo.h:18: DB::Connection::receivePacket() @ 0x0000000034c0f183 in /clickhouse
E           5. ./build_docker/./src/Client/ClientBase.cpp:1063: DB::ClientBase::receiveAndProcessPacket(std::shared_ptr<DB::IAST>, bool) @ 0x0000000034b83093 in /clickhouse
E           6. ./build_docker/./src/Client/ClientBase.cpp:1038: DB::ClientBase::receiveResult(std::shared_ptr<DB::IAST>, int, bool) @ 0x0000000034b82546 in /clickhouse
E           7. ./build_docker/./src/Client/ClientBase.cpp:0: DB::ClientBase::processOrdinaryQuery(String const&, std::shared_ptr<DB::IAST>) @ 0x0000000034b7fb96 in /clickhouse
E           8. ./build_docker/./src/Client/ClientBase.cpp:0: DB::ClientBase::processParsedSingleQuery(String const&, String const&, std::shared_ptr<DB::IAST>, std::optional<bool>, bool) @ 0x0000000034b7b6aa in /clickhouse
E           9. ./build_docker/./src/Client/ClientBase.cpp:2053: DB::ClientBase::executeMultiQuery(String const&) @ 0x0000000034b99c40 in /clickhouse
E           10. ./build_docker/./src/Client/ClientBase.cpp:0: DB::ClientBase::processQueryText(String const&) @ 0x0000000034b9cd54 in /clickhouse
E           11. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::ClientBase::runNonInteractive() @ 0x0000000034ba72f9 in /clickhouse
E           12. ./build_docker/./programs/client/Client.cpp:0: DB::Client::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001d9fb5dc in /clickhouse
E           13. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000003dce6207 in /clickhouse
E           14. ./build_docker/./programs/client/Client.cpp:0: mainEntryClickHouseClient(int, char**) @ 0x000000001da2dd7c in /clickhouse
E           15. ./build_docker/./programs/main.cpp:0: main @ 0x000000000bd67eb5 in /clickhouse
E           16. ? @ 0x00007f371ab22d90 in ?
E           17. ? @ 0x00007f371ab22e40 in ?
E           18. _start @ 0x000000000bc8f12e in /clickhouse
E            (version 23.8.16.41.altinitystable (altinity build))
E           (query: SELECT * FROM mydb.filtered_table1)

helpers/client.py:239: QueryRuntimeException
---------------------------- Captured stdout setup -----------------------------
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
---------------------------- Captured stderr setup -----------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
------------------------------ Captured log setup ------------------------------
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Stdout:1 (cluster.py:113, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : No running containers (conftest.py:60, cleanup_environment)
2024-09-24 00:20:32 [ 367 ] DEBUG : Pruning Docker networks (conftest.py:62, cleanup_environment)
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:['docker network prune --force'] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:105, run_and_check)
2024-09-24 00:20:32 [ 367 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:113, run_and_check)
2024-09-24 00:20:32 [ 367 ] INFO : Running tests in /ClickHouse/tests/integration/test_row_policy/test.py (cluster.py:2637, start)
2024-09-24 00:20:32 [ 367 ] DEBUG : Cluster start called. is_up=False (cluster.py:2644, start)
2024-09-24 00:20:32 [ 367 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 42
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker networks for project roottestrowpolicy are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker containers for project roottestrowpolicy are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker volumes for project roottestrowpolicy are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:20:32 [ 367 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 42
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker networks for project roottestrowpolicy are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:20:32 [ 367 ] DEBUG : Docker containers for project roottestrowpolicy are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:20:33 [ 367 ] DEBUG : Docker volumes for project roottestrowpolicy are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:docker container list --all --filter name='^/roottestrowpolicy_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : No running containers for project: roottestrowpolicy (cluster.py:782, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Stderr:Error response from daemon: a prune operation is already running (cluster.py:115, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Stdout:1 (cluster.py:113, run_and_check)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup directory for instance: node (cluster.py:2657, start)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4309, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for common tests configuration (cluster.py:4314, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy common configuration from helpers (cluster.py:4334, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Generate and write macros file (cluster.py:4353, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_row_policy/configs/config.d/remote_servers.xml'] to /ClickHouse/tests/integration/test_row_policy/_instances_0/node/configs/config.d (cluster.py:4383, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node/database (cluster.py:4400, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node/logs (cluster.py:4411, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4487, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup directory for instance: node2 (cluster.py:2657, start)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4309, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Create directory for common tests configuration (cluster.py:4314, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy common configuration from helpers (cluster.py:4334, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Generate and write macros file (cluster.py:4353, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_row_policy/configs/config.d/remote_servers.xml'] to /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/configs/config.d (cluster.py:4383, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/database (cluster.py:4400, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/logs (cluster.py:4411, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4487, create_dir)
2024-09-24 00:20:33 [ 367 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_row_policy/_instances_0/.env (cluster.py:78, _create_env_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:20:33 [ 367 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:546, _make_request)
2024-09-24 00:20:33 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'pull'] (cluster.py:105, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo2  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling zoo3  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Stderr:Pulling node  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:44 [ 367 ] DEBUG : Setup ZooKeeper (cluster.py:2699, start)
2024-09-24 00:20:44 [ 367 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination'] (cluster.py:2700, start)
2024-09-24 00:20:44 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:105, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /compose/docker_compose_keeper.yml (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:docker-py version: <module 'docker.version' from '/usr/local/lib/python3.10/dist-packages/docker/version.py'> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-119-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': '1.7.22', 'Details': {'GitCommit': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c'}}, {'Name': 'runc', 'Version': '1.1.14', 'Details': {'GitCommit': 'v1.1.14-0-g2c9f560'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-119-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:                      'ID': '7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Containers': 2, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.network.ensure: Creating network "roottestrowpolicy_default" with the default driver (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='roottestrowpolicy_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': 'da700a446e5d138cd7a1476f2cce92496fdc1a4343f493d9f229838c2717d45e', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {<Service: zoo3>, <Service: zoo2>, <Service: zoo1>} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo3> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo2> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo3_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo3', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo3', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo1> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo2', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo2', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestrowpolicy', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 01c5d05649c34834ec7a89763669f921de701a6ee114c6a489f91553f7c07658 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/lib/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '01c5d05649c34834ec7a89763669f921de701a6ee114c6a489f91553f7c07658'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 3f50bb25fb81f816e4dce31f9d66f0e9cc9c3efa697fed552a15d3029592ca9b (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/lib/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '3f50bb25fb81f816e4dce31f9d66f0e9cc9c3efa697fed552a15d3029592ca9b'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo1_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestrowpolicy', service='zoo1', number=1)} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestrowpolicy', service='zoo1', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 8aa383da29ffec3344f661d9e6b5b6fb9f0a0d021963da648b0982ab8b451851 (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestrowpolicy_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Target': '/var/log/clickhouse-keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553', user='0', volumes={}, name='roottestrowpolicy_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestrowpolicy', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '8aa383da29ffec3344f661d9e6b5b6fb9f0a0d021963da648b0982ab8b451851'}, host_config={'NetworkMode': 'roottestrowpolicy_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_row_policy/_instances_0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestrowpolicy_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', 'roottestrowpolicy_default') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232', 'roottestrowpolicy_default', aliases=['zoo3', '43c2a2b5df56'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217', 'roottestrowpolicy_default', aliases=['zoo2', '620722d4c0be'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a', 'roottestrowpolicy_default', aliases=['49e31cff24df', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('620722d4c0bee78a20a4d8f60d463dd5e7119f53691aa1b29a8fd0fa990ef217') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('43c2a2b5df563d212f44d295b205edbbc203de0a17972b20de9662cef4144232') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('49e31cff24df2355000294c423ae00596a8cac346e7e976bee77deca623fea9a') (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo3', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo3_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo3> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo2', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo2> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestrowpolicy', service='zoo1', number=1) (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_zoo1_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo1> (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2024-09-24 00:20:46 [ 367 ] DEBUG : Wait ZooKeeper to start (cluster.py:2337, wait_zookeeper_to_start)
2024-09-24 00:20:46 [ 367 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:46 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:46 [ 367 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.8.4, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:46 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:46 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:46 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:46 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:47 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:47 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:47 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:47 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:48 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:48 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:48 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:48 [ 367 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:51 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.8.2, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:51 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:51 [ 367 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.8.3, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2024-09-24 00:20:51 [ 367 ] INFO : Connecting to 172.16.8.3(172.16.8.3):2181, use_ssl: False (connection.py:650, _connect)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2024-09-24 00:20:51 [ 367 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2024-09-24 00:20:51 [ 367 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2024-09-24 00:20:51 [ 367 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2024-09-24 00:20:51 [ 367 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2024-09-24 00:20:51 [ 367 ] DEBUG : All instances of ZooKeeper started (cluster.py:2345, wait_zookeeper_to_start)
2024-09-24 00:20:51 [ 367 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_row_policy/_instances_0/.env --project-name roottestrowpolicy --file /ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml --file /compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml up -d --no-recreate') (cluster.py:2980, start)
2024-09-24 00:20:51 [ 367 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/.env', '--project-name', 'roottestrowpolicy', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_row_policy/_instances_0/node2/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:105, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node2_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node_1  ...  (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node_1  ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : Stderr:Creating roottestrowpolicy_node2_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:20:52 [ 367 ] DEBUG : ClickHouse instance created (cluster.py:2988, start)
2024-09-24 00:20:52 [ 367 ] DEBUG : get_instance_ip instance_name=node (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : Waiting for ClickHouse start in node, ip: 172.16.8.6... (cluster.py:2995, start)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:52 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:53 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : ClickHouse node started (cluster.py:2999, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:1971, get_instance_ip)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.8.5... (cluster.py:2995, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestrowpolicy_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : http://localhost:None "GET /v1.42/containers/2a77f6b1142771672473e46b34a80545b31bd944363e4f8292aea651670f699a/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:20:54 [ 367 ] DEBUG : ClickHouse node2 started (cluster.py:2999, start)
2024-09-24 00:20:54 [ 367 ] DEBUG : Executing query 
                CREATE DATABASE mydb;

                CREATE TABLE mydb.filtered_table1 (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table1 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.table (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.table values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.filtered_table2 (a UInt8, b UInt8, c UInt8, d UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table2 values (0, 0, 0, 0), (1, 2, 3, 4), (4, 3, 2, 1), (0, 0, 6, 0);

                CREATE TABLE mydb.filtered_table3 (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table3 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.`.filtered_table4` (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.`.filtered_t on node (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query 
                CREATE DATABASE mydb;

                CREATE TABLE mydb.filtered_table1 (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table1 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.table (a UInt8, b UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.table values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.filtered_table2 (a UInt8, b UInt8, c UInt8, d UInt8) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table2 values (0, 0, 0, 0), (1, 2, 3, 4), (4, 3, 2, 1), (0, 0, 6, 0);

                CREATE TABLE mydb.filtered_table3 (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.filtered_table3 values (0, 0), (0, 1), (1, 0), (1, 1);

                CREATE TABLE mydb.`.filtered_table4` (a UInt8, b UInt8, c UInt16 ALIAS a + b) ENGINE MergeTree ORDER BY a;
                INSERT INTO mydb.`.filtered_t on node2 (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query INSERT INTO mydb.local values (2, 0), (2, 1), (1, 0), (1, 1) on node (cluster.py:3423, query)
2024-09-24 00:20:55 [ 367 ] DEBUG : Executing query INSERT INTO mydb.local values (3, 0), (3, 1), (1, 0), (1, 1) on node2 (cluster.py:3423, query)
------------------------------ Captured log call -------------------------------
2024-09-24 00:20:55 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:55 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SYSTEM RELOAD CONFIG on node (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node2_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:20:56 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node2_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SYSTEM RELOAD CONFIG on node2 (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query SHOW POLICIES on node (cluster.py:3423, query)
2024-09-24 00:20:56 [ 367 ] DEBUG : Executing query CREATE USER X, Y on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query GRANT SELECT ON mydb.filtered_table1 TO X, Y on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query CREATE POLICY pA ON mydb.filtered_table1 FOR SELECT USING a >= b AS permissive TO X on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query SELECT * FROM mydb.filtered_table1 on node (cluster.py:3423, query)
2024-09-24 00:20:57 [ 367 ] DEBUG : Executing query SELECT * FROM mydb.filtered_table1 on node (cluster.py:3423, query)
______________________ test_tags_with_db_and_table_names _______________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_tags_with_db_and_table_names():
>       copy_policy_xml("tags_with_db_and_table_names.xml")

test_row_policy/test.py:796: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...DwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8dGFibGUgbmFtZT0nZmlsdGVyZWRfdGFibGUyJz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jICZndDsgZCArIDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L3RhYmxlPgogICAgICAgICAgICAgICAgPC9teWRiPgoKICAgICAgICAgICAgICAgIDxteWRiPgogICAgICAgICAgICAgICAgICAgIDx0YWJsZT4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5hID0gMDwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CgogICAgICAgICAgICAgICAgICAgIDx0YWJsZSBuYW1lPScuZmlsdGVyZWRfdGFibGU0Jz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMjwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CiAgICAgICAgICAgICAgICA8L215ZGI+CgogICAgICAgICAgICAgICAgPGRhdGFiYXNlIG5hbWU9J215ZGInPgogICAgICAgICAgICAgICAgICAgIDxmaWx0ZXJlZF90YWJsZTM+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YyA9IDA8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgIDwvZGF0YWJhc2U+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8dGFibGUgbmFtZT0nZmlsdGVyZWRfdGFibGUyJz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jICZndDsgZCArIDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L3RhYmxlPgogICAgICAgICAgICAgICAgPC9teWRiPgoKICAgICAgICAgICAgICAgIDxteWRiPgogICAgICAgICAgICAgICAgICAgIDx0YWJsZT4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5hID0gMDwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CgogICAgICAgICAgICAgICAgICAgIDx0YWJsZSBuYW1lPScuZmlsdGVyZWRfdGFibGU0Jz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMjwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CiAgICAgICAgICAgICAgICA8L215ZGI+CgogICAgICAgICAgICAgICAgPGRhdGFiYXNlIG5hbWU9J215ZGInPgogICAgICAgICAgICAgICAgICAgIDxmaWx0ZXJlZF90YWJsZTM+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YyA9IDA8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgIDwvZGF0YWJhc2U+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8dGFibGUgbmFtZT0nZmlsdGVyZWRfdGFibGUyJz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jICZndDsgZCArIDU8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L3RhYmxlPgogICAgICAgICAgICAgICAgPC9teWRiPgoKICAgICAgICAgICAgICAgIDxteWRiPgogICAgICAgICAgICAgICAgICAgIDx0YWJsZT4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5hID0gMDwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CgogICAgICAgICAgICAgICAgICAgIDx0YWJsZSBuYW1lPScuZmlsdGVyZWRfdGFibGU0Jz4KICAgICAgICAgICAgICAgICAgICAgICAgPGZpbHRlcj5jID0gMjwvZmlsdGVyPgogICAgICAgICAgICAgICAgICAgIDwvdGFibGU+CiAgICAgICAgICAgICAgICA8L215ZGI+CgogICAgICAgICAgICAgICAgPGRhdGFiYXNlIG5hbWU9J215ZGInPgogICAgICAgICAgICAgICAgICAgIDxmaWx0ZXJlZF90YWJsZTM+CiAgICAgICAgICAgICAgICAgICAgICAgIDxmaWx0ZXI+YyA9IDA8L2ZpbHRlcj4KICAgICAgICAgICAgICAgICAgICA8L2ZpbHRlcmVkX3RhYmxlMz4KICAgICAgICAgICAgICAgIDwvZGF0YWJhc2U+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_________ test_throwif_error_in_prewhere_with_same_condition_as_filter _________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_throwif_error_in_prewhere_with_same_condition_as_filter():
>       copy_policy_xml("normal_filter2_table2.xml")

test_row_policy/test.py:289: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...DwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:13 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:13 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:13 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
__________ test_throwif_error_in_where_with_same_condition_as_filter ___________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_throwif_error_in_where_with_same_condition_as_filter():
>       copy_policy_xml("normal_filter2_table2.xml")

test_row_policy/test.py:282: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...DwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
____________ test_throwif_in_prewhere_doesnt_expose_restricted_data ____________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_throwif_in_prewhere_doesnt_expose_restricted_data():
>       copy_policy_xml("no_filters.xml")

test_row_policy/test.py:308: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_____________ test_throwif_in_where_doesnt_expose_restricted_data ______________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_throwif_in_where_doesnt_expose_restricted_data():
>       copy_policy_xml("no_filters.xml")

test_row_policy/test.py:296: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:14 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:14 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:14 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
__________________________ test_users_xml_is_readonly __________________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_users_xml_is_readonly():
>       assert re.search(
            "storage is readonly",
            node.query_and_get_error("DROP POLICY default ON mydb.filtered_table1"),
        )
E       AssertionError: assert None
E        +  where None = <function search at 0x7f264477ca60>('storage is readonly', 'Code: 209. DB::NetException: Timeout: connect timed out: 172.16.8.6:9000 (172.16.8.6:9000, connection timeout 10000 ms). (SOCKET_TIMEOUT), Stack trace (when copying this message, always include the lines below):\n\n0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e077d33 in /clickhouse\n1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5c5dd0 in /clickhouse\n2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String const&, long>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type, std::type_identity<long>::type>, String&&, String const&, long&&) @ 0x0000000034c14241 in /clickhouse\n3. ./build_docker/./src/Client/Connection.cpp:218: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x0000000034bf347f in /clickhouse\n4. ./build_docker/./src/Client/Connection.cpp:423: DB::Connection::getServerVersion(DB::ConnectionTimeouts const&, String&, unsigned long&, unsigned long&, unsigned long&, unsigned long&) @ 0x0000000034bf9d1b in /clickhouse\n5. ./build_docker/./programs/client/Client.cpp:446: DB::Client::connect() @ 0x000000001d9fe9a2 in /clickhouse\n6. ./build_docker/./programs/client/Client.cpp:0: DB::Client::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001d9fb33e in /clickhouse\n7. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000003dce6207 in /clickhouse\n8. ./build_docker/./programs/client/Client.cpp:0: mainEntryClickHouseClient(int, char**) @ 0x000000001da2dd7c in /clickhouse\n9. ./build_docker/./programs/main.cpp:0: main @ 0x000000000bd67eb5 in /clickhouse\n10. ? @ 0x00007f29c997ed90 in ?\n11. ? @ 0x00007f29c997ee40 in ?\n12. _start @ 0x000000000bc8f12e in /clickhouse\n\n\n')
E        +    where <function search at 0x7f264477ca60> = re.search
E        +    and   'Code: 209. DB::NetException: Timeout: connect timed out: 172.16.8.6:9000 (172.16.8.6:9000, connection timeout 10000 ms). (SOCKET_TIMEOUT), Stack trace (when copying this message, always include the lines below):\n\n0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e077d33 in /clickhouse\n1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5c5dd0 in /clickhouse\n2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::NetException::NetException<String, String const&, long>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String const&>::type, std::type_identity<long>::type>, String&&, String const&, long&&) @ 0x0000000034c14241 in /clickhouse\n3. ./build_docker/./src/Client/Connection.cpp:218: DB::Connection::connect(DB::ConnectionTimeouts const&) @ 0x0000000034bf347f in /clickhouse\n4. ./build_docker/./src/Client/Connection.cpp:423: DB::Connection::getServerVersion(DB::ConnectionTimeouts const&, String&, unsigned long&, unsigned long&, unsigned long&, unsigned long&) @ 0x0000000034bf9d1b in /clickhouse\n5. ./build_docker/./programs/client/Client.cpp:446: DB::Client::connect() @ 0x000000001d9fe9a2 in /clickhouse\n6. ./build_docker/./programs/client/Client.cpp:0: DB::Client::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001d9fb33e in /clickhouse\n7. ./build_docker/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000003dce6207 in /clickhouse\n8. ./build_docker/./programs/client/Client.cpp:0: mainEntryClickHouseClient(int, char**) @ 0x000000001da2dd7c in /clickhouse\n9. ./build_docker/./programs/main.cpp:0: main @ 0x000000000bd67eb5 in /clickhouse\n10. ? @ 0x00007f29c997ed90 in ?\n11. ? @ 0x00007f29c997ee40 in ?\n12. _start @ 0x000000000bc8f12e in /clickhouse\n\n\n' = <bound method ClickHouseInstance.query_and_get_error of <helpers.cluster.ClickHouseInstance object at 0x7f263a274b50>>('DROP POLICY default ON mydb.filtered_table1')
E        +      where <bound method ClickHouseInstance.query_and_get_error of <helpers.cluster.ClickHouseInstance object at 0x7f263a274b50>> = <helpers.cluster.ClickHouseInstance object at 0x7f263a274b50>.query_and_get_error

test_row_policy/test.py:725: AssertionError
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:15 [ 367 ] DEBUG : Executing query DROP POLICY default ON mydb.filtered_table1 on node (cluster.py:3495, query_and_get_error)
______________________________ test_with_prewhere ______________________________
[gw2] linux -- Python 3.10.12 /usr/bin/python3

    def test_with_prewhere():
>       copy_policy_xml("normal_filter2_table2.xml")

test_row_policy/test.py:229: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_row_policy/test.py:36: in copy_policy_xml
    current_node.copy_file_to_container(
helpers/cluster.py:3973: in copy_file_to_container
    return self.cluster.copy_file_to_container(
helpers/cluster.py:2049: in copy_file_to_container
    self.exec_in_container(
helpers/cluster.py:2012: in exec_in_container
    result = subprocess_check_call(
helpers/cluster.py:158: in subprocess_check_call
    return run_and_check(args, detach=detach, nothrow=nothrow)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF...DwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml']
env = None, shell = False, stdout = -1, stderr = -1, timeout = 300
nothrow = False, detach = False

    def run_and_check(
        args,
        env=None,
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        timeout=300,
        nothrow=False,
        detach=False,
    ):
        if detach:
            subprocess.Popen(
                args,
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL,
                env=env,
                shell=shell,
            )
            return
    
        logging.debug(f"Command:{args}")
        res = subprocess.run(
            args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout
        )
        out = res.stdout.decode("utf-8")
        err = res.stderr.decode("utf-8")
        # check_call(...) from subprocess does not print stderr, so we do it manually
        for outline in out.splitlines():
            logging.debug(f"Stdout:{outline}")
        for errline in err.splitlines():
            logging.debug(f"Stderr:{errline}")
        if res.returncode != 0:
            logging.debug(f"Exitcode:{res.returncode}")
            if env:
                logging.debug(f"Env:{env}")
            if not nothrow:
>               raise Exception(
                    f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}"
                )
E               Exception: Command ['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] return non-zero code 1: Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running

helpers/cluster.py:121: Exception
------------------------------ Captured log call -------------------------------
2024-09-24 00:21:26 [ 367 ] DEBUG : run container_id:roottestrowpolicy_node_1 detach:False nothrow:False cmd: ['bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:2004, exec_in_container)
2024-09-24 00:21:26 [ 367 ] DEBUG : Command:['docker', 'exec', 'roottestrowpolicy_node_1', 'bash', '-c', 'echo PGNsaWNraG91c2U+CiAgICA8dXNlcnM+CiAgICAgICAgPGRlZmF1bHQ+CiAgICAgICAgICAgIDwhLS0gRm9yIHRlc3RpbmcgdGhlIHRhYmxlIGZpbHRlcnMgLS0+CiAgICAgICAgICAgIDxkYXRhYmFzZXM+CiAgICAgICAgICAgICAgICA8bXlkYj4KICAgICAgICAgICAgICAgICAgICA8IS0tIFNpbXBsZSBleHByZXNzaW9uIGZpbHRlciAtLT4KICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyZWRfdGFibGUyPgogICAgICAgICAgICAgICAgICAgICAgICA8ZmlsdGVyPmEgJmd0OyAwPC9maWx0ZXI+CiAgICAgICAgICAgICAgICAgICAgPC9maWx0ZXJlZF90YWJsZTI+CiAgICAgICAgICAgICAgICA8L215ZGI+CiAgICAgICAgICAgIDwvZGF0YWJhc2VzPgogICAgICAgIDwvZGVmYXVsdD4KICAgIDwvdXNlcnM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/row_policy.xml'] (cluster.py:105, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Stderr:Error response from daemon: Container b9c1dbcf91d5b6e091665f15514a96aa8719ca7494c85c0c70b355d9f0176681 is not running (cluster.py:115, run_and_check)
2024-09-24 00:21:26 [ 367 ] DEBUG : Exitcode:1 (cluster.py:117, run_and_check)
_______________________________ test_persistence _______________________________
[gw6] linux -- Python 3.10.12 /usr/bin/python3

    def test_persistence():
        create_function_query1 = "CREATE FUNCTION MySum1 AS (a, b) -> a + b"
        create_function_query2 = "CREATE FUNCTION MySum2 AS (a, b) -> MySum1(a, b) + b"
    
        instance.query(create_function_query1)
        instance.query(create_function_query2)
    
        assert instance.query("SELECT MySum1(1,2)") == "3\n"
        assert instance.query("SELECT MySum2(1,2)") == "5\n"
    
        instance.restart_clickhouse()
    
        assert instance.query("SELECT MySum1(1,2)") == "3\n"
        assert instance.query("SELECT MySum2(1,2)") == "5\n"
    
        instance.query("DROP FUNCTION MySum2")
        instance.query("DROP FUNCTION MySum1")
    
        instance.restart_clickhouse()
    
>       assert "Unknown function MySum1" in instance.query_and_get_error(
            "SELECT MySum1(1, 2)"
        )
E       assert 'Unknown function MySum1' in "Received exception from server (version 23.8.16):\nCode: 46. DB::Exception: Received from 172.16.19.2:9000. DB::Exception: Function with name 'MySum1' does not exists. In scope SELECT MySum1(1, 2). Stack trace:\n\n0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e077d33 in /usr/bin/clickhouse\n1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5c5dd0 in /usr/bin/clickhouse\n2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String, String>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&, String&&, String&&) @ 0x00000000278582bb in /usr/bin/clickhouse\n3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5064: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000312734ea in /usr/bin/clickhouse\n4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003122c32e in /usr/bin/clickhouse\n5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5688: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003122a047 in /usr/bin/clickhouse\n6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5877: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000031243591 in /usr/bin/clickhouse\n7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000031216eb4 in /usr/bin/clickhouse\n8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>, std::shared_ptr<DB::Context const>) @ 0x0000000031211176 in /usr/bin/clickhouse\n9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:183: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003120c288 in /usr/bin/clickhouse\n10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x00000000315f4012 in /usr/bin/clickhouse\n11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:161: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&) @ 0x00000000315f0029 in /usr/bin/clickhouse\n12. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x00000000314421d5 in /usr/bin/clickhouse\n13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320e429b in /usr/bin/clickhouse\n14. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320dc301 in /usr/bin/clickhouse\n15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ecd2fd in /usr/bin/clickhouse\n16. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f0a0d0 in /usr/bin/clickhouse\n17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcac08f in /usr/bin/clickhouse\n18. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcace57 in /usr/bin/clickhouse\n19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e18c67c in /usr/bin/clickhouse\n20. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e185728 in /usr/bin/clickhouse\n21. ? @ 0x00007faf1cf35ac3 in ?\n22. ? @ 0x00007faf1cfc7850 in ?\n. (UNKNOWN_FUNCTION)\n(query: SELECT MySum1(1, 2))\n"
E        +  where "Received exception from server (version 23.8.16):\nCode: 46. DB::Exception: Received from 172.16.19.2:9000. DB::Exception: Function with name 'MySum1' does not exists. In scope SELECT MySum1(1, 2). Stack trace:\n\n0. ./build_docker/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000003e077d33 in /usr/bin/clickhouse\n1. ./build_docker/./src/Common/Exception.cpp:98: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001d5c5dd0 in /usr/bin/clickhouse\n2. ./build_docker/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<String&, String, String>(int, FormatStringHelperImpl<std::type_identity<String&>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&, String&&, String&&) @ 0x00000000278582bb in /usr/bin/clickhouse\n3. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5064: DB::(anonymous namespace)::QueryAnalyzer::resolveFunction(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x00000000312734ea in /usr/bin/clickhouse\n4. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNode(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003122c32e in /usr/bin/clickhouse\n5. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5688: DB::(anonymous namespace)::QueryAnalyzer::resolveExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&, bool, bool) @ 0x000000003122a047 in /usr/bin/clickhouse\n6. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:5877: DB::(anonymous namespace)::QueryAnalyzer::resolveProjectionExpressionNodeList(std::shared_ptr<DB::IQueryTreeNode>&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000031243591 in /usr/bin/clickhouse\n7. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::(anonymous namespace)::QueryAnalyzer::resolveQuery(std::shared_ptr<DB::IQueryTreeNode> const&, DB::(anonymous namespace)::IdentifierResolveScope&) @ 0x0000000031216eb4 in /usr/bin/clickhouse\n8. ./build_docker/./src/Analyzer/Passes/QueryAnalysisPass.cpp:0: DB::QueryAnalysisPass::run(std::shared_ptr<DB::IQueryTreeNode>, std::shared_ptr<DB::Context const>) @ 0x0000000031211176 in /usr/bin/clickhouse\n9. ./build_docker/./src/Analyzer/QueryTreePassManager.cpp:183: DB::QueryTreePassManager::run(std::shared_ptr<DB::IQueryTreeNode>) @ 0x000000003120c288 in /usr/bin/clickhouse\n10. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:0: DB::(anonymous namespace)::buildQueryTreeAndRunPasses(std::shared_ptr<DB::IAST> const&, DB::SelectQueryOptions const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<DB::IStorage> const&) @ 0x00000000315f4012 in /usr/bin/clickhouse\n11. ./build_docker/./src/Interpreters/InterpreterSelectQueryAnalyzer.cpp:161: DB::InterpreterSelectQueryAnalyzer::InterpreterSelectQueryAnalyzer(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context const> const&, DB::SelectQueryOptions const&) @ 0x00000000315f0029 in /usr/bin/clickhouse\n12. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x00000000314421d5 in /usr/bin/clickhouse\n13. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000320e429b in /usr/bin/clickhouse\n14. ./build_docker/./src/Interpreters/executeQuery.cpp:1224: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000320dc301 in /usr/bin/clickhouse\n15. ./build_docker/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x0000000034ecd2fd in /usr/bin/clickhouse\n16. ./build_docker/./src/Server/TCPHandler.cpp:2161: DB::TCPHandler::run() @ 0x0000000034f0a0d0 in /usr/bin/clickhouse\n17. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003dcac08f in /usr/bin/clickhouse\n18. ./build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x000000003dcace57 in /usr/bin/clickhouse\n19. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x000000003e18c67c in /usr/bin/clickhouse\n20. ./build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003e185728 in /usr/bin/clickhouse\n21. ? @ 0x00007faf1cf35ac3 in ?\n22. ? @ 0x00007faf1cfc7850 in ?\n. (UNKNOWN_FUNCTION)\n(query: SELECT MySum1(1, 2))\n" = <bound method ClickHouseInstance.query_and_get_error of <helpers.cluster.ClickHouseInstance object at 0x7f72cc7186d0>>('SELECT MySum1(1, 2)')
E        +    where <bound method ClickHouseInstance.query_and_get_error of <helpers.cluster.ClickHouseInstance object at 0x7f72cc7186d0>> = <helpers.cluster.ClickHouseInstance object at 0x7f72cc7186d0>.query_and_get_error

test_user_defined_object_persistence/test.py:38: AssertionError
---------------------------- Captured stdout setup -----------------------------
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
---------------------------- Captured stderr setup -----------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
------------------------------ Captured log setup ------------------------------
2024-09-24 00:22:31 [ 379 ] INFO : Running tests in /ClickHouse/tests/integration/test_user_defined_object_persistence/test.py (cluster.py:2637, start)
2024-09-24 00:22:31 [ 379 ] DEBUG : Cluster start called. is_up=False (cluster.py:2644, start)
2024-09-24 00:22:31 [ 379 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 25
  Running: 25
  Paused: 0
  Stopped: 0
 Images: 43
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker networks for project roottestuserdefinedobjectpersistence are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker containers for project roottestuserdefinedobjectpersistence are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker volumes for project roottestuserdefinedobjectpersistence are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 28
  Running: 25
  Paused: 0
  Stopped: 3
 Images: 43
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker networks for project roottestuserdefinedobjectpersistence are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker containers for project roottestuserdefinedobjectpersistence are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Docker volumes for project roottestuserdefinedobjectpersistence are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:22:31 [ 379 ] DEBUG : Command:docker container list --all --filter name='^/roottestuserdefinedobjectpersistence_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:22:31 [ 379 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : No running containers for project: roottestuserdefinedobjectpersistence (cluster.py:782, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:22:31 [ 379 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2024-09-24 00:22:31 [ 379 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:22:31 [ 379 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:22:31 [ 379 ] DEBUG : Stdout:10 (cluster.py:113, run_and_check)
2024-09-24 00:22:31 [ 379 ] DEBUG : Setup directory for instance: instance (cluster.py:2657, start)
2024-09-24 00:22:31 [ 379 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4309, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Create directory for common tests configuration (cluster.py:4314, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Copy common configuration from helpers (cluster.py:4334, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Generate and write macros file (cluster.py:4353, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/configs/config.d (cluster.py:4383, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/database (cluster.py:4400, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/logs (cluster.py:4411, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4487, create_dir)
2024-09-24 00:22:31 [ 379 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env (cluster.py:78, _create_env_file)
2024-09-24 00:22:31 [ 379 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:22:31 [ 379 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:22:31 [ 379 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2024-09-24 00:22:31 [ 379 ] DEBUG : No config file found (config.py:28, find_config_file)
2024-09-24 00:22:31 [ 379 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:546, _make_request)
2024-09-24 00:22:31 [ 379 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env', '--project-name', 'roottestuserdefinedobjectpersistence', '--file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/docker-compose.yml', 'pull'] (cluster.py:105, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Pulling instance ...  (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Pulling instance ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Pulling instance ... digest: sha256:3c12bcb41068a0596b... (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Pulling instance ... status: image is up to date for a... (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Pulling instance ... done (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env --project-name roottestuserdefinedobjectpersistence --file /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/docker-compose.yml up -d --no-recreate') (cluster.py:2980, start)
2024-09-24 00:22:43 [ 379 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env', '--project-name', 'roottestuserdefinedobjectpersistence', '--file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:105, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Creating network "roottestuserdefinedobjectpersistence_default" with the default driver (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Creating roottestuserdefinedobjectpersistence_instance_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : Stderr:Creating roottestuserdefinedobjectpersistence_instance_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:22:43 [ 379 ] DEBUG : ClickHouse instance created (cluster.py:2988, start)
2024-09-24 00:22:43 [ 379 ] DEBUG : get_instance_ip instance_name=instance (cluster.py:1971, get_instance_ip)
2024-09-24 00:22:43 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestuserdefinedobjectpersistence_instance_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:43 [ 379 ] DEBUG : Waiting for ClickHouse start in instance, ip: 172.16.19.2... (cluster.py:2995, start)
2024-09-24 00:22:43 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestuserdefinedobjectpersistence_instance_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:43 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:44 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:45 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:46 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:46 [ 379 ] DEBUG : http://localhost:None "GET /v1.42/containers/d11aeba2f00c5fdab525a7ff038b3bb0ee3ad7088649aa4586af9cc92b97c65d/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2024-09-24 00:22:46 [ 379 ] DEBUG : ClickHouse instance started (cluster.py:2999, start)
------------------------------ Captured log call -------------------------------
2024-09-24 00:22:46 [ 379 ] DEBUG : Executing query CREATE FUNCTION MySum1 AS (a, b) -> a + b on instance (cluster.py:3423, query)
2024-09-24 00:22:46 [ 379 ] DEBUG : Executing query CREATE FUNCTION MySum2 AS (a, b) -> MySum1(a, b) + b on instance (cluster.py:3423, query)
2024-09-24 00:22:46 [ 379 ] DEBUG : Executing query SELECT MySum1(1,2) on instance (cluster.py:3423, query)
2024-09-24 00:22:46 [ 379 ] DEBUG : Executing query SELECT MySum2(1,2) on instance (cluster.py:3423, query)
2024-09-24 00:22:47 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:47 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:22:47 [ 379 ] DEBUG : Stdout:    PID TTY          TIME CMD (cluster.py:113, run_and_check)
2024-09-24 00:22:47 [ 379 ] DEBUG : Stdout:      8 ?        00:00:02 clickhouse (cluster.py:113, run_and_check)
2024-09-24 00:22:47 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill  clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:47 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'pkill  clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:22:47 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:47 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:47 [ 379 ] DEBUG : Stdout:8 (cluster.py:113, run_and_check)
2024-09-24 00:22:48 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:48 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:48 [ 379 ] DEBUG : Stdout:8 (cluster.py:113, run_and_check)
2024-09-24 00:22:49 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:49 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:49 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:49 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:49 [ 379 ] DEBUG : No clickhouse process running. Start new one. (cluster.py:3775, start_clickhouse)
2024-09-24 00:22:49 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:49 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', '0', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:105, run_and_check)
2024-09-24 00:22:50 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:50 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:50 [ 379 ] DEBUG : Stdout:342 (cluster.py:113, run_and_check)
2024-09-24 00:22:50 [ 379 ] DEBUG : Clickhouse process running. (cluster.py:3783, start_clickhouse)
2024-09-24 00:22:50 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:50 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:50 [ 379 ] DEBUG : Stdout:342 (cluster.py:113, run_and_check)
2024-09-24 00:22:50 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:51 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:52 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:52 [ 379 ] DEBUG : Executing query SELECT MySum1(1,2) on instance (cluster.py:3423, query)
2024-09-24 00:22:52 [ 379 ] DEBUG : Executing query SELECT MySum2(1,2) on instance (cluster.py:3423, query)
2024-09-24 00:22:52 [ 379 ] DEBUG : Executing query DROP FUNCTION MySum2 on instance (cluster.py:3423, query)
2024-09-24 00:22:53 [ 379 ] DEBUG : Executing query DROP FUNCTION MySum1 on instance (cluster.py:3423, query)
2024-09-24 00:22:53 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:53 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:22:53 [ 379 ] DEBUG : Stdout:    PID TTY          TIME CMD (cluster.py:113, run_and_check)
2024-09-24 00:22:53 [ 379 ] DEBUG : Stdout:    342 ?        00:00:02 clickhouse (cluster.py:113, run_and_check)
2024-09-24 00:22:53 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill  clickhouse'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:53 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'pkill  clickhouse'] (cluster.py:105, run_and_check)
2024-09-24 00:22:53 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:53 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:53 [ 379 ] DEBUG : Stdout:342 (cluster.py:113, run_and_check)
2024-09-24 00:22:54 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:54 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:54 [ 379 ] DEBUG : Stdout:342 (cluster.py:113, run_and_check)
2024-09-24 00:22:55 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:55 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:55 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:55 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:55 [ 379 ] DEBUG : No clickhouse process running. Start new one. (cluster.py:3775, start_clickhouse)
2024-09-24 00:22:55 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:55 [ 379 ] DEBUG : Command:['docker', 'exec', '-u', '0', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon'] (cluster.py:105, run_and_check)
2024-09-24 00:22:56 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:56 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:56 [ 379 ] DEBUG : Stdout:701 (cluster.py:113, run_and_check)
2024-09-24 00:22:56 [ 379 ] DEBUG : Clickhouse process running. (cluster.py:3783, start_clickhouse)
2024-09-24 00:22:56 [ 379 ] DEBUG : run container_id:roottestuserdefinedobjectpersistence_instance_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2004, exec_in_container)
2024-09-24 00:22:56 [ 379 ] DEBUG : Command:['docker', 'exec', 'roottestuserdefinedobjectpersistence_instance_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:105, run_and_check)
2024-09-24 00:22:56 [ 379 ] DEBUG : Stdout:701 (cluster.py:113, run_and_check)
2024-09-24 00:22:56 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:57 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:58 [ 379 ] DEBUG : Executing query select 20 on instance (cluster.py:3423, query)
2024-09-24 00:22:58 [ 379 ] DEBUG : Executing query SELECT MySum1(1, 2) on instance (cluster.py:3495, query_and_get_error)
--------------------------- Captured stderr teardown ---------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
---------------------------- Captured log teardown -----------------------------
2024-09-24 00:22:59 [ 379 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env', '--project-name', 'roottestuserdefinedobjectpersistence', '--file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:105, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Stderr:Stopping roottestuserdefinedobjectpersistence_instance_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Stderr:Stopping roottestuserdefinedobjectpersistence_instance_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/.env', '--project-name', 'roottestuserdefinedobjectpersistence', '--file', '/ClickHouse/tests/integration/test_user_defined_object_persistence/_instances_0/instance/docker-compose.yml', 'down', '--volumes'] (cluster.py:105, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Stderr:Removing roottestuserdefinedobjectpersistence_instance_1 ...  (cluster.py:115, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Stderr:Removing roottestuserdefinedobjectpersistence_instance_1 ... done (cluster.py:115, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Stderr:Removing network roottestuserdefinedobjectpersistence_default (cluster.py:115, run_and_check)
2024-09-24 00:23:00 [ 379 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2024-09-24 00:23:00 [ 379 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 20
  Running: 20
  Paused: 0
  Stopped: 0
 Images: 43
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-119-generic
 Operating System: Ubuntu 22.04.5 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a34ef02984a1
 ID: 23c7ffba-fa59-4a62-b9f4-6a037892fec4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2024-09-24 00:23:01 [ 379 ] DEBUG : Docker networks for project roottestuserdefinedobjectpersistence are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2024-09-24 00:23:01 [ 379 ] DEBUG : Docker containers for project roottestuserdefinedobjectpersistence are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2024-09-24 00:23:01 [ 379 ] DEBUG : Docker volumes for project roottestuserdefinedobjectpersistence are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2024-09-24 00:23:01 [ 379 ] DEBUG : Command:docker container list --all --filter name='^/roottestuserdefinedobjectpersistence_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2024-09-24 00:23:01 [ 379 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : No running containers for project: roottestuserdefinedobjectpersistence (cluster.py:782, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2024-09-24 00:23:01 [ 379 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2024-09-24 00:23:01 [ 379 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2024-09-24 00:23:01 [ 379 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2024-09-24 00:23:01 [ 379 ] DEBUG : Stdout:7 (cluster.py:113, run_and_check)
=============================== warnings summary ===============================
test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy
  /usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host '172.16.10.5'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
    warnings.warn(

test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
test_zookeeper_config/test_secure.py::test_secure_connection
  /usr/local/lib/python3.10/dist-packages/kazoo/handlers/utils.py:225: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated
    context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================== slowest durations ===============================
135.19s call     test_undrop_query/test.py::test_undrop_drop_and_undrop_loop
123.05s call     test_storage_hudi/test.py::test_multiple_hudi_files
37.77s call     test_zookeeper_config/test_secure.py::test_secure_connection
35.77s setup    test_storage_s3_queue/test.py::test_delete_after_processing[ordered]
31.96s call     test_version_update_after_mutation/test.py::test_mutate_and_upgrade
31.53s setup    test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy
29.99s teardown test_throttling/test.py::test_write_throttling[user_remote_throttling]
29.92s call     test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]
29.62s call     test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]
28.58s setup    test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue
27.24s setup    test_storage_redis/test.py::test_create_table
26.45s call     test_version_update_after_mutation/test.py::test_upgrade_while_mutation
24.73s teardown test_storage_url_with_proxy/test.py::test_s3_with_proxy_list
23.71s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy
23.62s setup    test_truncate_database/test_distributed.py::test_truncate_database_distributed
23.38s setup    test_version_update_after_mutation/test.py::test_mutate_and_upgrade
23.29s setup    test_row_policy/test.py::test_some_users_without_policies
23.16s teardown test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage
23.16s setup    test_zookeeper_config/test_secure.py::test_secure_connection
22.51s teardown test_storage_hudi/test.py::test_types
22.40s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]
22.26s setup    test_settings_constraints_distributed/test.py::test_insert_clamps_settings
21.72s setup    test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]
21.53s call     test_throttling/test.py::test_backup_throttling_override
21.31s setup    test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]
21.12s setup    test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]
20.68s call     test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]
19.35s setup    test_storage_hudi/test.py::test_multiple_hudi_files
19.16s setup    test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage
18.88s setup    test_throttling/test.py::test_backup_throttling[no_local_throttling]
17.75s setup    test_session_log/test.py::test_grpc_session
17.18s call     test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]
16.82s call     test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]
16.80s call     test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]
16.76s call     test_row_policy/test.py::test_some_users_without_policies
15.94s setup    test_transactions/test.py::test_rollback_unfinished_on_restart1
15.42s setup    test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]
15.25s call     test_throttling/test.py::test_write_throttling[user_remote_throttling]
15.19s setup    test_system_queries/test.py::test_DROP_DNS_CACHE
15.12s setup    test_user_defined_object_persistence/test.py::test_persistence
14.50s call     test_throttling/test.py::test_write_throttling[local_user_throttling]
14.42s setup    test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access
13.49s call     test_throttling/test.py::test_backup_throttling[user_local_throttling]
13.49s call     test_throttling/test.py::test_read_throttling[user_remote_throttling]
13.48s call     test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]
13.44s call     test_throttling/test.py::test_read_throttling[user_local_throttling]
13.34s setup    test_storage_url_with_proxy/test.py::test_s3_with_proxy_list
12.69s call     test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]
12.63s call     test_user_defined_object_persistence/test.py::test_persistence
12.19s teardown test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]
11.02s call     test_throttling/test.py::test_write_throttling[server_remote_throttling]
10.51s call     test_row_policy/test.py::test_users_xml_is_readonly
10.14s call     test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]
9.95s teardown test_truncate_database/test_distributed.py::test_truncate_database_distributed
9.56s call     test_throttling/test.py::test_read_throttling[server_remote_throttling]
9.49s call     test_throttling/test.py::test_read_throttling[server_local_throttling]
9.39s teardown test_zookeeper_config/test_secure.py::test_secure_connection
9.25s call     test_throttling/test.py::test_write_throttling[local_server_throttling]
9.07s call     test_throttling/test.py::test_backup_throttling[server_local_throttling]
8.05s call     test_transactions/test.py::test_rollback_unfinished_on_restart1
7.89s call     test_transactions/test.py::test_rollback_unfinished_on_restart2
7.79s teardown test_throttling/test.py::test_write_throttling[no_local_throttling]
7.74s teardown test_throttling/test.py::test_write_throttling[server_remote_throttling]
7.73s teardown test_throttling/test.py::test_write_throttling[local_user_throttling]
7.23s teardown test_throttling/test.py::test_write_throttling[no_remote_throttling]
7.18s teardown test_throttling/test.py::test_read_throttling[server_local_throttling]
7.15s teardown test_throttling/test.py::test_write_throttling[local_server_throttling]
7.11s teardown test_throttling/test.py::test_read_throttling[server_remote_throttling]
7.04s teardown test_throttling/test.py::test_read_throttling[user_local_throttling]
6.84s setup    test_undrop_query/test.py::test_undrop_drop_and_undrop_loop
6.80s teardown test_version_update_after_mutation/test.py::test_upgrade_while_mutation
6.77s teardown test_row_policy/test.py::test_with_prewhere
6.74s teardown test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]
6.57s call     test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]
6.53s setup    test_settings_constraints_distributed/test.py::test_select_clamps_settings
6.26s teardown test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]
6.25s teardown test_throttling/test.py::test_backup_throttling[server_local_throttling]
6.13s teardown test_throttling/test.py::test_read_throttling[user_remote_throttling]
6.10s teardown test_throttling/test.py::test_backup_throttling_override
6.07s teardown test_throttling/test.py::test_backup_throttling[user_local_throttling]
6.05s teardown test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]
6.03s teardown test_throttling/test.py::test_read_throttling[no_local_throttling]
6.01s teardown test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]
6.00s teardown test_throttling/test.py::test_read_throttling[no_remote_throttling]
5.98s teardown test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]
5.97s call     test_system_queries/test.py::test_system_flush_logs
5.85s teardown test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue
5.52s teardown test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]
5.42s teardown test_throttling/test.py::test_backup_throttling[no_local_throttling]
5.42s teardown test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]
5.34s teardown test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]
4.58s teardown test_storage_redis/test.py::test_update
4.44s teardown test_session_log/test.py::test_postgres_session
4.43s call     test_settings_constraints_distributed/test.py::test_select_clamps_settings
4.23s teardown test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access
4.22s teardown test_settings_constraints_distributed/test.py::test_select_clamps_settings
4.20s teardown test_transactions/test.py::test_rollback_unfinished_on_restart2
4.01s call     test_storage_hudi/test.py::test_types
3.97s call     test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue
3.86s teardown test_undrop_query/test.py::test_undrop_drop_and_undrop_loop
3.70s call     test_storage_s3_queue/test.py::test_max_set_age
3.68s call     test_session_log/test.py::test_parallel_sessions
3.33s teardown test_system_queries/test.py::test_system_flush_logs
3.23s call     test_storage_hudi/test.py::test_single_hudi_file
2.90s call     test_storage_s3_queue/test.py::test_direct_select_file[ordered]
2.64s call     test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY
2.53s call     test_settings_constraints_distributed/test.py::test_insert_clamps_settings
2.52s call     test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]
2.36s call     test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list
2.28s call     test_storage_redis/test.py::test_create_table
2.28s teardown test_user_defined_object_persistence/test.py::test_persistence
2.25s teardown test_system_flush_logs/test.py::test_system_suspend
2.24s call     test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]
2.23s call     test_system_flush_logs/test.py::test_system_suspend
2.21s call     test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access
2.19s call     test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]
2.17s call     test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]
2.15s call     test_system_queries/test.py::test_DROP_DNS_CACHE
2.15s call     test_storage_s3_queue/test.py::test_direct_select_file[unordered]
1.83s call     test_session_log/test.py::test_postgres_session
1.81s call     test_truncate_database/test_distributed.py::test_truncate_database_distributed
1.77s call     test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]
1.73s call     test_storage_redis/test.py::test_delete
1.73s call     test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]
1.65s call     test_session_log/test.py::test_grpc_session
1.47s call     test_storage_redis/test.py::test_simple_insert
1.43s call     test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch
1.41s call     test_storage_s3_queue/test.py::test_max_set_size
1.37s call     test_session_log/test.py::test_mysql_session
1.30s call     test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy
1.29s call     test_storage_s3_queue/test.py::test_failed_retry[ordered]
1.25s call     test_storage_redis/test.py::test_update
1.14s call     test_storage_s3_queue/test.py::test_failed_retry[unordered]
1.06s call     test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy
0.98s call     test_storage_redis/test.py::test_truncate
0.93s setup    test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]
0.87s call     test_storage_redis/test.py::test_simple_select
0.84s call     test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]
0.83s call     test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]
0.82s call     test_storage_redis/test.py::test_select_int
0.76s setup    test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch
0.70s call     test_storage_s3_queue/test.py::test_delete_after_processing[ordered]
0.65s call     test_storage_s3_queue/test.py::test_delete_after_processing[unordered]
0.63s call     test_throttling/test.py::test_backup_throttling[no_local_throttling]
0.63s call     test_system_flush_logs/test.py::test_system_logs_non_empty_queue
0.59s call     test_throttling/test.py::test_read_throttling[no_remote_throttling]
0.59s call     test_storage_url_with_proxy/test.py::test_s3_with_proxy_list
0.58s call     test_throttling/test.py::test_write_throttling[no_remote_throttling]
0.54s call     test_throttling/test.py::test_read_throttling[no_local_throttling]
0.53s call     test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]
0.49s call     test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS
0.48s call     test_throttling/test.py::test_write_throttling[no_local_throttling]
0.47s call     test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]
0.47s setup    test_storage_s3_queue/test.py::test_max_set_size
0.44s setup    test_storage_s3_queue/test.py::test_direct_select_file[unordered]
0.44s setup    test_storage_s3_queue/test.py::test_direct_select_file[ordered]
0.43s setup    test_storage_s3_queue/test.py::test_delete_after_processing[unordered]
0.42s setup    test_system_flush_logs/test.py::test_system_logs[system.text_log-0]
0.42s setup    test_system_flush_logs/test.py::test_system_logs[system.query_log-1]
0.42s setup    test_system_flush_logs/test.py::test_system_logs[system.part_log-1]
0.40s setup    test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]
0.40s setup    test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]
0.39s setup    test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]
0.38s setup    test_storage_s3_queue/test.py::test_max_set_age
0.38s setup    test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]
0.35s setup    test_storage_s3_queue/test.py::test_failed_retry[ordered]
0.34s setup    test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]
0.34s setup    test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]
0.34s setup    test_storage_s3_queue/test.py::test_failed_retry[unordered]
0.27s call     test_system_flush_logs/test.py::test_system_logs[system.text_log-0]
0.27s call     test_system_flush_logs/test.py::test_system_logs[system.query_log-1]
0.22s call     test_system_flush_logs/test.py::test_system_logs[system.part_log-1]
0.22s setup    test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]
0.22s setup    test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]
0.22s call     test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage
0.22s call     test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]
0.22s call     test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]
0.04s teardown test_row_policy/test.py::test_users_xml_is_readonly
0.02s call     test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data
0.02s call     test_row_policy/test.py::test_tags_with_db_and_table_names
0.02s teardown test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data
0.02s teardown test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter
0.02s teardown test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data
0.02s teardown test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter
0.02s call     test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter
0.02s call     test_row_policy/test.py::test_with_prewhere
0.02s teardown test_row_policy/test.py::test_tags_with_db_and_table_names
0.02s call     test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter
0.02s teardown test_row_policy/test.py::test_some_users_without_policies
0.02s call     test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data
0.00s setup    test_session_log/test.py::test_postgres_session
0.00s setup    test_version_update_after_mutation/test.py::test_upgrade_while_mutation
0.00s teardown test_version_update_after_mutation/test.py::test_mutate_and_upgrade
0.00s teardown test_storage_hudi/test.py::test_multiple_hudi_files
0.00s teardown test_storage_s3_queue/test.py::test_delete_after_processing[ordered]
0.00s setup    test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]
0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]
0.00s setup    test_throttling/test.py::test_backup_throttling[user_local_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[user_local_throttling]
0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]
0.00s setup    test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[no_local_throttling]
0.00s teardown test_transactions/test.py::test_rollback_unfinished_on_restart1
0.00s setup    test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[user_remote_throttling]
0.00s setup    test_throttling/test.py::test_write_throttling[server_remote_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]
0.00s setup    test_throttling/test.py::test_write_throttling[no_local_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[no_remote_throttling]
0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]
0.00s setup    test_throttling/test.py::test_write_throttling[no_remote_throttling]
0.00s setup    test_throttling/test.py::test_write_throttling[local_server_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[server_local_throttling]
0.00s setup    test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]
0.00s setup    test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data
0.00s setup    test_throttling/test.py::test_backup_throttling[server_local_throttling]
0.00s setup    test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY
0.00s setup    test_throttling/test.py::test_write_throttling[local_user_throttling]
0.00s setup    test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy
0.00s setup    test_row_policy/test.py::test_users_xml_is_readonly
0.00s setup    test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter
0.00s teardown test_settings_constraints_distributed/test.py::test_insert_clamps_settings
0.00s setup    test_throttling/test.py::test_write_throttling[user_remote_throttling]
0.00s setup    test_throttling/test.py::test_read_throttling[server_remote_throttling]
0.00s setup    test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data
0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.text_log-0]
0.00s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy
0.00s setup    test_system_flush_logs/test.py::test_system_logs_non_empty_queue
0.00s setup    test_throttling/test.py::test_backup_throttling_override
0.00s setup    test_storage_hudi/test.py::test_single_hudi_file
0.00s setup    test_row_policy/test.py::test_with_prewhere
0.00s setup    test_row_policy/test.py::test_tags_with_db_and_table_names
0.00s setup    test_system_flush_logs/test.py::test_system_suspend
0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]
0.00s setup    test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter
0.00s setup    test_storage_redis/test.py::test_simple_select
0.00s setup    test_storage_redis/test.py::test_truncate
0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]
0.00s teardown test_storage_s3_queue/test.py::test_max_set_size
0.00s teardown test_system_queries/test.py::test_DROP_DNS_CACHE
0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]
0.00s teardown test_storage_redis/test.py::test_simple_select
0.00s setup    test_storage_hudi/test.py::test_types
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.part_log-1]
0.00s teardown test_storage_s3_queue/test.py::test_delete_after_processing[unordered]
0.00s teardown test_system_flush_logs/test.py::test_system_logs_non_empty_queue
0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[ordered]
0.00s setup    test_transactions/test.py::test_rollback_unfinished_on_restart2
0.00s teardown test_session_log/test.py::test_parallel_sessions
0.00s teardown test_storage_s3_queue/test.py::test_direct_select_file[ordered]
0.00s teardown test_storage_redis/test.py::test_create_table
0.00s teardown test_storage_redis/test.py::test_delete
0.00s teardown test_storage_redis/test.py::test_select_int
0.00s setup    test_storage_redis/test.py::test_select_int
0.00s teardown test_session_log/test.py::test_grpc_session
0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]
0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]
0.00s setup    test_storage_redis/test.py::test_simple_insert
0.00s setup    test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list
0.00s teardown test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]
0.00s teardown test_storage_s3_queue/test.py::test_direct_select_file[unordered]
0.00s teardown test_system_flush_logs/test.py::test_system_logs[system.query_log-1]
0.00s setup    test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS
0.00s setup    test_storage_redis/test.py::test_delete
0.00s setup    test_session_log/test.py::test_parallel_sessions
0.00s teardown test_storage_s3_queue/test.py::test_max_set_age
0.00s setup    test_storage_redis/test.py::test_update
0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]
0.00s teardown test_storage_hudi/test.py::test_single_hudi_file
0.00s setup    test_session_log/test.py::test_mysql_session
0.00s teardown test_storage_redis/test.py::test_simple_insert
0.00s teardown test_storage_redis/test.py::test_truncate
0.00s setup    test_system_queries/test.py::test_system_flush_logs
0.00s teardown test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS
0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[unordered]
0.00s teardown test_session_log/test.py::test_mysql_session
0.00s teardown test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY
=========================== short test summary info ============================
FAILED test_settings_constraints_distributed/test.py::test_select_clamps_settings
FAILED test_row_policy/test.py::test_some_users_without_policies - helpers.cl...
FAILED test_row_policy/test.py::test_tags_with_db_and_table_names - Exception...
FAILED test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter
FAILED test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter
FAILED test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data
FAILED test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data
FAILED test_row_policy/test.py::test_users_xml_is_readonly - AssertionError: ...
FAILED test_row_policy/test.py::test_with_prewhere - Exception: Command ['doc...
FAILED test_user_defined_object_persistence/test.py::test_persistence - asser...
ERROR test_row_policy/test.py::test_some_users_without_policies - Exception: ...
ERROR test_row_policy/test.py::test_tags_with_db_and_table_names - Exception:...
ERROR test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter
ERROR test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter
ERROR test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data
ERROR test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data
ERROR test_row_policy/test.py::test_users_xml_is_readonly - Exception: Comman...
ERROR test_row_policy/test.py::test_with_prewhere - Exception: Command ['dock...
PASSED test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]
PASSED test_system_flush_logs/test.py::test_system_logs[system.part_log-1]
PASSED test_system_flush_logs/test.py::test_system_logs[system.query_log-1]
PASSED test_system_queries/test.py::test_DROP_DNS_CACHE
PASSED test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]
PASSED test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS
PASSED test_system_flush_logs/test.py::test_system_logs[system.text_log-0]
PASSED test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]
PASSED test_system_flush_logs/test.py::test_system_logs_non_empty_queue
PASSED test_session_log/test.py::test_grpc_session
PASSED test_throttling/test.py::test_backup_throttling[no_local_throttling]
PASSED test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY
PASSED test_session_log/test.py::test_mysql_session
PASSED test_system_flush_logs/test.py::test_system_suspend
PASSED test_session_log/test.py::test_parallel_sessions
PASSED test_settings_constraints_distributed/test.py::test_insert_clamps_settings
PASSED test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]
PASSED test_session_log/test.py::test_postgres_session
PASSED test_system_queries/test.py::test_system_flush_logs
PASSED test_storage_redis/test.py::test_create_table
PASSED test_storage_redis/test.py::test_delete
PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]
PASSED test_storage_redis/test.py::test_select_int
PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy
PASSED test_storage_redis/test.py::test_simple_insert
PASSED test_storage_redis/test.py::test_simple_select
PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list
PASSED test_storage_redis/test.py::test_truncate
PASSED test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy
PASSED test_storage_s3_queue/test.py::test_delete_after_processing[ordered]
PASSED test_storage_redis/test.py::test_update
PASSED test_storage_s3_queue/test.py::test_delete_after_processing[unordered]
PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]
PASSED test_storage_s3_queue/test.py::test_direct_select_file[ordered]
PASSED test_storage_s3_queue/test.py::test_direct_select_file[unordered]
PASSED test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]
PASSED test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]
PASSED test_storage_s3_queue/test.py::test_failed_retry[ordered]
PASSED test_storage_s3_queue/test.py::test_failed_retry[unordered]
PASSED test_throttling/test.py::test_backup_throttling[server_local_throttling]
PASSED test_storage_s3_queue/test.py::test_max_set_age
PASSED test_storage_s3_queue/test.py::test_max_set_size
PASSED test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch
PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue
PASSED test_transactions/test.py::test_rollback_unfinished_on_restart1
PASSED test_transactions/test.py::test_rollback_unfinished_on_restart2
PASSED test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]
PASSED test_storage_url_with_proxy/test.py::test_s3_with_proxy_list
PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]
PASSED test_version_update_after_mutation/test.py::test_mutate_and_upgrade
PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]
PASSED test_version_update_after_mutation/test.py::test_upgrade_while_mutation
PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]
PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]
PASSED test_zookeeper_config/test_secure.py::test_secure_connection
PASSED test_truncate_database/test_distributed.py::test_truncate_database_distributed
PASSED test_throttling/test.py::test_backup_throttling[user_local_throttling]
PASSED test_storage_hudi/test.py::test_multiple_hudi_files
PASSED test_storage_hudi/test.py::test_single_hudi_file
PASSED test_storage_hudi/test.py::test_types
PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]
PASSED test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]
PASSED test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]
PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]
PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]
PASSED test_undrop_query/test.py::test_undrop_drop_and_undrop_loop
PASSED test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]
PASSED test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]
PASSED test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]
PASSED test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]
PASSED test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access
PASSED test_throttling/test.py::test_backup_throttling_override
PASSED test_throttling/test.py::test_read_throttling[no_local_throttling]
PASSED test_throttling/test.py::test_read_throttling[no_remote_throttling]
PASSED test_throttling/test.py::test_read_throttling[server_local_throttling]
PASSED test_throttling/test.py::test_read_throttling[server_remote_throttling]
PASSED test_throttling/test.py::test_read_throttling[user_local_throttling]
PASSED test_throttling/test.py::test_read_throttling[user_remote_throttling]
PASSED test_throttling/test.py::test_write_throttling[local_server_throttling]
PASSED test_throttling/test.py::test_write_throttling[local_user_throttling]
PASSED test_throttling/test.py::test_write_throttling[no_local_throttling]
PASSED test_throttling/test.py::test_write_throttling[no_remote_throttling]
PASSED test_throttling/test.py::test_write_throttling[server_remote_throttling]
PASSED test_throttling/test.py::test_write_throttling[user_remote_throttling]
SKIPPED [1] test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py:29: Disabled for debug and sanitizers. Too slow.
== 10 failed, 84 passed, 1 skipped, 8 warnings, 8 errors in 453.70s (0:07:33) ==
Traceback (most recent call last):
  File "/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration/./runner", line 455, in <module>
    subprocess.check_call(cmd, shell=True)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'docker run   --rm --name clickhouse_integration_tests_g8i3jl --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos --volume=/run:/run/host:ro --mount type=bind,source=/home/ubuntu/_work/_temp/test/dockerd_volume_dir,target=/var/lib/docker -e DOCKER_HELPER_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_BASE_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_KERBERIZED_HADOOP_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_KERBEROS_KDC_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_JAVA_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_JS_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_MYSQL_PHP_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_NGINX_DAV_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=0-9992f5eba3591142ff413d31cb83a7bf4f0e2553   -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_NEW_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_row_policy/test.py::test_some_users_without_policies test_row_policy/test.py::test_tags_with_db_and_table_names test_row_policy/test.py::test_throwif_error_in_prewhere_with_same_condition_as_filter test_row_policy/test.py::test_throwif_error_in_where_with_same_condition_as_filter test_row_policy/test.py::test_throwif_in_prewhere_doesnt_expose_restricted_data test_row_policy/test.py::test_throwif_in_where_doesnt_expose_restricted_data test_row_policy/test.py::test_users_xml_is_readonly test_row_policy/test.py::test_with_prewhere test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_env_proxy test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_proxy_list test_s3_table_function_with_https_proxy/test.py::test_s3_with_https_remote_proxy test_s3_zero_copy_ttl/test_vertical_merge_memory_usage.py::test_vertical_merge_memory_usage test_session_log/test.py::test_grpc_session test_session_log/test.py::test_mysql_session test_session_log/test.py::test_parallel_sessions test_session_log/test.py::test_postgres_session test_settings_constraints_distributed/test.py::test_insert_clamps_settings test_settings_constraints_distributed/test.py::test_select_clamps_settings test_storage_hudi/test.py::test_multiple_hudi_files test_storage_hudi/test.py::test_single_hudi_file test_storage_hudi/test.py::test_types test_storage_redis/test.py::test_create_table test_storage_redis/test.py::test_delete test_storage_redis/test.py::test_select_int test_storage_redis/test.py::test_simple_insert test_storage_redis/test.py::test_simple_select test_storage_redis/test.py::test_truncate test_storage_redis/test.py::test_update 'test_storage_s3_queue/test.py::test_delete_after_processing[ordered]' 'test_storage_s3_queue/test.py::test_delete_after_processing[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_file[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 'test_system_flush_logs/test.py::test_system_logs[system.metric_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.part_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.query_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.query_thread_log-1]' 'test_system_flush_logs/test.py::test_system_logs[system.text_log-0]' 'test_system_flush_logs/test.py::test_system_logs[system.trace_log-1]' test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs 'test_throttling/test.py::test_backup_throttling[no_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]' test_throttling/test.py::test_backup_throttling_override 'test_throttling/test.py::test_read_throttling[no_local_throttling]' 'test_throttling/test.py::test_read_throttling[no_remote_throttling]' 'test_throttling/test.py::test_read_throttling[server_local_throttling]' 'test_throttling/test.py::test_read_throttling[server_remote_throttling]' 'test_throttling/test.py::test_read_throttling[user_local_throttling]' 'test_throttling/test.py::test_read_throttling[user_remote_throttling]' 'test_throttling/test.py::test_write_throttling[local_server_throttling]' 'test_throttling/test.py::test_write_throttling[local_user_throttling]' 'test_throttling/test.py::test_write_throttling[no_local_throttling]' 'test_throttling/test.py::test_write_throttling[no_remote_throttling]' 'test_throttling/test.py::test_write_throttling[server_remote_throttling]' 'test_throttling/test.py::test_write_throttling[user_remote_throttling]' test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 test_truncate_database/test_distributed.py::test_truncate_database_distributed test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_user_defined_object_persistence/test.py::test_persistence test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_zookeeper_config/test_secure.py::test_secure_connection  -vvv" altinityinfra/integration-tests-runner:0-9992f5eba3591142ff413d31cb83a7bf4f0e2553 ' returned non-zero exit status 1.